1. Building the hccl-test Image Download the dependency packages Python-3.8.18.tgz Ascend-cann-toolkit_8.0.RC2_linux-x86_64.run Ascend-cann-kernels-910b_8.0.RC2_linux.run mpich-3.2.1.tar.gz If downloading is inconvenient, you can also copy them straight out of the image I built. Write the Dockerfile 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 FROM ubuntu:22.

1. Building the nccl-test Image Check the CUDA version 1 2 3 nvidia-smi | grep "CUDA Version" | awk '{print $9}' 12.2 Write the Dockerfile 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 cat > Dockerfile << EOF FROM nvidia/cuda:12.1.0-cudnn8-devel-ubuntu22.04 ENV DEBIAN_FRONTEND=noninteractive ARG CONDA_VERSION WORKDIR /workspace ENV DEBIAN_FRONTEND=noninteractive RUN apt-get update && apt install -y openmpi-bin libopenmpi-dev ssh openssh-server net-tools vim git iputils-ping nfs-common RUN git clone https://github.

1. Introduction to Volcano Volcano is a Kubernetes-based resource scheduling system open-sourced by Huawei. Compared with the native scheduler, its notable features are: Supports gang scheduling Scheduling batch jobs easily runs into deadlock: for example, two jobs each need 10 Pods running simultaneously to start. When both jobs are submitted at the same time, it is possible that only part of the Pods of each are scheduled, so neither job can run properly and they wait on each other.