NVIDIA_VISIBLE_DEVICES Specifies the GPU devices visible to the program 1 CUDA_VISIBLE_DEVICES=0,1 Available values: 1,2, a comma-separated list of GPU UUIDs or indices all, all GPUs none, load the driver but with no GPU access void, do not load the driver NVIDIA_DRIVER_CAPABILITIES Controls which driver libraries/binaries will be installed inside the container

HCCL_IF_IP Configures the IP of the root communication NIC for HCCL initialization. Priority: environment variable HCCL_IF_IP > environment variable HCCL_SOCKET_IFNAME > NICs other than docker/lo (in ascending lexicographic order of NIC name) > docker NIC > lo NIC. 1 export HCCL_IF_IP=10.0.0.1 HCCL_IF_BASE_PORT Specifies the starting port number of the Host NIC.

1. Setting Environment Variables 1 2 export NAMESPACE=xxx export PVC=xxx 2. The Dataset Will Not Become Ready 2.1 Fluid Component Problems 1 kubectl -n fluid-system get pod -o wide | grep -v "Running" There may be cases where something did not start up properly. 2.2 Abnormally Terminated Datasets Abnormal resources can cause Fluid resources to restart repeatedly, and manual intervention is required to delete them.

1. Environment Preparation Download Miniforge 1 wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh" Install Miniforge 1 bash Miniforge3-$(uname)-$(uname -m).sh 1 2 echo "export PATH=$HOME/miniforge3/bin:$PATH" >> ~/.bashrc source ~/.bashrc Create the environment 1 conda create -n vllm python=3.12 vLLM currently requires Python 3.9+. Activate the environment 1 conda activate vllm Install dependencies 1 conda install

1. Building the Image To make testing convenient, the model files are packaged into the image here. Download the model 1 2 3 4 git clone https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat cd Qwen1.5-1.8B-Chat && git lfs pull rm -rf .git cd .. Write the Dockerfile 1 2 3 4 5 cat <<EOF > Dockerfile