차이
문서의 선택한 두 판 사이의 차이를 보여줍니다.
| 양쪽 이전 판이전 판다음 판 | 이전 판 |
| distributed_computing [2020/04/26 04:06] – rex8312 | distributed_computing [2024/03/23 02:38] (현재) – 바깥 편집 127.0.0.1 |
|---|
| ===== Distribued SGD ===== | ===== Distribued SGD ===== |
| |
| * [[https://www.microsoft.com/en-us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/|ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters, Microsoft Research]] | * [[https://www.microsoft.com/en-us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/|ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters]] |
| | * Microsoft, DeepSpeed, pytorch, Language Model |
| | * [[https://devblogs.nvidia.com/fast-multi-gpu-collectives-nccl/|NCCL]] |
| | * Nvidia |
| | * https://www.fast.ai/2018/08/10/fastai-diu-imagenet/ |
| | * [[http://seba1511.net/dist_blog/#ref-gorila|An Introduction to Distributed Deep Learning, 2016-12]] |
| |
| ===== Streaming processing ===== | ===== Streaming processing ===== |
| |
| * https://github.com/robinhood/faust | * https://github.com/robinhood/faust |
