1. Deployment Plan Before starting the containerized deployment, a few requirements: To simplify delivery, only one image is needed For reliability, deploy as many replicas as possible Start different services through different parameters Inject configuration through environment variables and render it into configuration files Below is the deployment plan for DeepSeek 3FS:

1. Introduction to the Qwen Model In April 2023, Alibaba released the beta version of Qwen. In December 2023, Alibaba open-sourced the first version of Qwen. In September 2024, Alibaba released Qwen2.5. In January 2025, Alibaba released Qwen 2.5-Max. Qwen 2.5 is the latest series of the Qwen large language

1. Environment Preparation Download Miniforge 1 wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh" Install Miniforge 1 bash Miniforge3-$(uname)-$(uname -m).sh 1 2 echo "export PATH=$HOME/miniforge3/bin:$PATH" >> ~/.bashrc source ~/.bashrc Create the environment 1 conda create -n vllm python=3.12 vLLM currently requires Python 3.9+. Activate the environment 1 conda activate vllm Install dependencies 1 conda install

1. Building the Image To make testing convenient, the model files are packaged into the image here. Download the model 1 2 3 4 git clone https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat cd Qwen1.5-1.8B-Chat && git lfs pull rm -rf .git cd .. Write the Dockerfile 1 2 3 4 5 cat <<EOF > Dockerfile

Embedding Models Embedding models map high-dimensional data into a lower-dimensional space, which makes the data easier to process and analyze. Text Models Here is a leaderboard, https://huggingface.co/spaces/mteb/leaderboard The leaderboard above gives each model’s score, parameter count, memory usage, vector dimension, maximum tokens, and other information. Below are some commonly used open-source models: