====== CUDA ======

  * [[http://tleyden.github.io/blog/2014/10/25/cuda-6-dot-5-on-aws-gpu-instance-running-ubuntu-14-dot-04/]]
    * AWS에 CUDA 6.5 설치

환경변수로 CUDA device 설정

<code python>
# CUDA 사용금지
os.environ['CUDA_VISIBLE_DEVICES'] = ""
# 또는 
os.environ['CUDA_VISIBLE_DEVICES'] = "-1"

# 첫 번째 디바이스만 보임
os.environ['CUDA_VISIBLE_DEVICES'] = "0"
</code>

===== Mixed precision =====

  * https://devblogs.nvidia.com/apex-pytorch-easy-mixed-precision-training/
  * https://devblogs.nvidia.com/mixed-precision-training-deep-neural-networks/
===== CUDA 관련 도구 =====

=== nvidia-smi ===
nvidia 디바이스 정보 보여주는 도구
windows에서는 "C:\Program Files\NVIDIA Corporation\NVSMI"에 있음


<code>
nvidia-smi -l  # 주기적으로 디바이스 정보를 출력
nvidia-smi -lms T  # T ms 마다 디바이스 정보를 출력
nvidia-smi dmon  # 실시간으로 gpu, 메모리 사용, 온도 출력
</code>

=== nvidia driver 설치 ===

드라이버 삭제

  nvidia-installer --uninstall
  
최신 버전 드라이버 설치

  nvidia-installer --update

  * http://www.linuxquestions.org/questions/slackware-14/cannot-reinstall-nvidia-driver-module-being-loaded-wtf-296828/


=== 팬 속도 조절 ===

  sudo nvidia-xconfig
  sudo nvidia-xconfig --cool-bits=4
  
  # command line에서 조절
  ## 읽기
  nvidia-settings -q [gpu:0]/GPUCoreTemp -c :0.0
  nvidia-settings -q [fan:0]/GPUCurrentFanSpeed -c :0.0
  ## 쓰기
  nvidia-settings -a [gpu:0]/GPUFanControlState=1 -a [fan:0]/GPUTargetFanSpeed=75 -c :0.0
  
  ## 멀티 GPU
  nvidia-settings -a "[gpu:0]/GPUFanControlState=1" -a "[fan:0]/GPUCurrentFanSpeed=n" \
                -a "[gpu:1]/GPUFanControlState=1" -a  [fan:1]/GPUCurrentFanSpeed=n" &

  * http://askubuntu.com/questions/42494/how-can-i-change-the-nvidia-gpu-fan-speed
  * https://bbs.archlinux.org/viewtopic.php?id=196716
  * http://blog.cryptohaze.com/2011/02/scripts-to-control-nvidia-fan-speeds.html
  * https://github.com/MisterPup/Nvidia-Dynamic-Fan-Control
  * https://wiki.archlinux.org/index.php/NVIDIA/Tips_and_tricks



===== AWS + Docker + CUDA =====

  * http://tleyden.github.io/blog/2014/10/25/docker-on-aws-gpu-ubuntu-14-dot-04-slash-cuda-6-dot-5/

===== Docker =====

  * https://github.com/NVIDIA/nvidia-docker


===== Headless GPU의 fan 속도 조절 ======

  * https://sites.google.com/site/akohlmey/random-hacks/nvidia-gpu-coolness
  * https://github.com/boris-dimitrov/set_gpu_fans_public
  * https://gist.github.com/clarkwinkelmann/13af7a328fb20b9a7d78
  * http://thomascomputerrepair.com/nvidia-settings-kde.php

===== GPU 모니터링 및 작업할당 =====

  * https://medium.com/mlreview/using-your-idle-deep-learning-hardware-for-mining-c1b9887491fa


===== python =====

  * https://towardsdatascience.com/python-performance-and-gpus-1be860ffd58d?ncid=so-twi-n2-96487&linkId=100000006881312


===== 라이브러리 ======

  * cuDF: GPU DataFrames
    * https://github.com/rapidsai/cudf
  * cuML: scikit-learn과 유사한 ML 라이브러리
    * https://github.com/rapidsai/cuml
  * cuGraph: 그래프 라이브러리
    * https://github.com/rapidsai/cugraph
  * Rapids: https://rapids.ai/start.html