Out of the Box

사용자 도구

사이트 도구

추적:
distributed_computing

차이

문서의 선택한 두 판 사이의 차이를 보여줍니다.

차이 보기로 링크

양쪽 이전 판이전 판
다음 판		이전 판
distributed_computing [2020/04/26 04:07] – [Distribued SGD] rex8312distributed_computing [2024/03/23 02:38] (현재) – 바깥 편집 127.0.0.1
줄 21: 줄 21:
   * [[https://www.microsoft.com/en-us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/|ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters]]   * [[https://www.microsoft.com/en-us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/|ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters]]
     * Microsoft, DeepSpeed, pytorch, Language Model     * Microsoft, DeepSpeed, pytorch, Language Model
 +  * [[https://devblogs.nvidia.com/fast-multi-gpu-collectives-nccl/|NCCL]]
 +    * Nvidia
 +  * https://www.fast.ai/2018/08/10/fastai-diu-imagenet/
 +  * [[http://seba1511.net/dist_blog/#ref-gorila|An Introduction to Distributed Deep Learning, 2016-12]]
  
 ===== Streaming processing ===== ===== Streaming processing =====
  
   * https://github.com/robinhood/faust   * https://github.com/robinhood/faust
distributed_computing.1587874040.txt.gz · 마지막으로 수정됨: (바깥 편집)