Using a vLLM Application to Verify an Inference Node
· ☕ 1 min read
1. Building the Image To make testing convenient, the model files are packaged into the image here. Download the model 1 2 3 4 git clone https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat cd Qwen1.5-1.8B-Chat && git lfs pull rm -rf .git cd .. Write the Dockerfile 1 2 3 4 5 cat <<EOF > Dockerfile