목차
Distributed Computing
Distribued SGD
Streaming processing
Distributed Computing
Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks, 2020-02
OpenAI, NeuralMMO
Dota 2 with Large Scale Deep Reinforcement Learning, 2019-12
OpenAI, OpenAI Five, Dota
PyTorch Multi-GPU 제대로 학습하기-당근마켓
http://mpi4py.scipy.org/docs/usrman/index.html
https://ray.readthedocs.io/en/latest/
https://ray-project.github.io/
https://modin.readthedocs.io/en/latest/
https://arrow.apache.org/docs/index.html
https://arrow.apache.org/
https://ray-project.github.io/2017/10/15/fast-python-serialization-with-ray-and-arrow.html
Distribued SGD
ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters
Microsoft, DeepSpeed, pytorch, Language Model
NCCL
Nvidia
https://www.fast.ai/2018/08/10/fastai-diu-imagenet/
An Introduction to Distributed Deep Learning, 2016-12
Streaming processing
https://github.com/robinhood/faust