====== 2021-01 ZeRO-Offload: Democratizing Billion-Scale Model Training ======

  * https://arxiv.org/abs/2101.06840
  * https://www.deepspeed.ai/
  * https://github.com/microsoft/DeepSpeed

{{tag>zero-offload zero deepspeed offloading memory_efficient Microsoft "Jie Ren" "Yuxiong He" 2021}}