1. What the Image Tag Suffixes Mean
base/cuda: includes the CUDA runtime
runtime: builds on base, adding the CUDA math libraries and the NCCL and cuDNN runtimes
devel: builds on runtime, adding headers and the development tools used to build CUDA images; particularly useful for multi-stage builds
cuddn: builds on the above, adding the cuDNN neural network acceleration library
py3: a Python 3 environment
2. CUDA Images
| Image | AMD64 image size | ARM64 image size |
|---|
| nvidia/cuda:12.3.2-base-ubuntu22.04 | 87.01 MB | 30.82 MB |
| nvidia/cuda:12.3.2-runtime-ubuntu22.04 | 1.28 GB | 1.23 GB |
| nvidia/cuda:12.3.2-devel-ubuntu22.04 | 3.68 GB | 3.14 GB |
| nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04 | 1.91 GB | 1.86 GB |
| nvidia/cuda:12.3.2-cudnn9-devel-ubuntu22.04 | 4.31 GB | 3.77 GB |
| nvidia/cuda:12.3.2-base-ubuntu20.04 | 88.25 MB | 32.56 MB |
| nvidia/cuda:12.3.2-runtime-ubuntu20.04 | 1.28 GB | 1.23 GB |
| nvidia/cuda:12.3.2-devel-ubuntu20.04 | 3.66 GB | 3.12 GB |
| nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu20.04 | 1.91 GB | 1.86 GB |
| nvidia/cuda:12.3.2-cudnn9-devel-ubuntu20.04 | 4.29 GB | 3.75 GB |
3. PyTorch Images
The official PyTorch images provided by PyTorch.
| Image | AMD64 image size |
|---|
| pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime | 3.44 GB |
| pytorch/pytorch:2.2.2-cuda12.1-cudnn8-devel | 8.47 GB |
4. NVIDIA PyTorch Images
PyTorch images packaged by NVIDIA, including cuda, cuBlas, cuDNN, OpenMPI, TensorRT, and other packages.
5. Running an Image
1
2
3
4
5
| docker run --security-opt apparmor=unconfined \
--security-opt seccomp=unconfined \
--gpus all \
-it --rm \
pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime bash
|
- Specify the number of GPUs
1
2
3
4
5
| docker run --security-opt apparmor=unconfined \
--security-opt seccomp=unconfined \
--gpus 3 \
-it --rm \
pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime bash
|
1
2
3
4
5
| docker run --security-opt apparmor=unconfined \
--security-opt seccomp=unconfined \
--gpus '"device=0,1"' \
-it --rm \
pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime bash
|
Instead of GPU indices you can also use UUIDs. UUIDs can be queried with nvidia-smi -L.
- Set the shared memory (shm)
1
2
3
4
5
6
| docker run --security-opt apparmor=unconfined \
--security-opt seccomp=unconfined \
--gpus all \
--shm-size=64g \
-it --rm \
pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime bash
|
- Set system resource limits
1
2
3
4
5
6
| docker run --security-opt apparmor=unconfined \
--security-opt seccomp=unconfined \
--gpus all \
--ulimit memlock=-1 --ulimit stack=67108864 \
-it --rm \
pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime bash
|
1
2
3
4
5
6
| docker run --security-opt apparmor=unconfined \
--security-opt seccomp=unconfined \
--gpus all \
--ipc=host \
-it --rm \
pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime bash
|
- Mount the current directory into the container’s /workspace
1
2
3
4
5
6
| docker run --security-opt apparmor=unconfined \
--security-opt seccomp=unconfined \
--gpus all \
-v "$PWD":/workspace \
-it --rm \
pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime bash
|
- Set CPU and memory limits
1
2
3
4
5
6
| docker run --security-opt apparmor=unconfined \
--security-opt seccomp=unconfined \
--gpus all \
--cpus=2 --memory=4g \
-it --rm \
pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime bash
|
1
2
3
4
5
6
| docker run --security-opt apparmor=unconfined \
--security-opt seccomp=unconfined \
--gpus all \
--privileged \
-it --rm \
pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime bash
|
It is worth noting that a container started in Docker privileged mode can use all GPU resources, which makes the device list specified by --gpus ineffective.
- Start with maximum resources
1
2
3
4
5
6
7
8
9
| docker run --security-opt apparmor=unconfined \
--security-opt seccomp=unconfined \
--gpus all \
--ulimit memlock=-1 --ulimit stack=67108864 \
-v "$PWD":/workspace \
--privileged \
--shm-size=64g \
-it --rm \
pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime bash
|
