Using LLaMA-Factory to Fine-Tune the Qwen3 Model

1. Testing the Original Model

Start the environment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
nerdctl run -it \
        --security-opt apparmor=unconfined \
        --security-opt seccomp=unconfined \
        --gpus all \
        --ipc=host \
        -p 8000:8000 \
        --ulimit memlock=-1 \
        --ulimit stack=67108864 \
        --name vllm \
        --volume /data/models:/data/models \
        --entrypoint /bin/bash \
        vllm/vllm-openai:v0.10.1.1

The vllm-related tests below are all based on this environment.

Start the service

1
2
3
4
5
6
export CUDA_VISIBLE_DEVICES=0
python3 -m vllm.entrypoints.openai.api_server \
  --model /data/models/Qwen3-0.6B \
  --served-model-name /data/models/Qwen3-0.6B \
  --host 0.0.0.0 \
  --port 8000

Test

1
2
3
4
5
6
7
8
curl -X POST "http://127.0.0.1:8000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "/data/models/Qwen3-0.6B",
    "messages": [
      {"role": "user", "content": "介绍一下你自己/no_think"}
    ]
  }'

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
{
  "id": "chatcmpl-e831f557a8384a9b833e8033f7ac8ffa",
  "object": "chat.completion",
  "created": 1758524426,
  "model": "/data/models/Qwen3-0.6B",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "<think>\n\n</think>\n\n您好！我是小明，来自中国。很高兴和您交流。如果您有任何问题或需要帮助，请随时告诉我！",
        "refusal": null,
        "annotations": null,
        "audio": null,
        "function_call": null,
        "tool_calls": [],
        "reasoning_content": null
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "prompt_tokens": 13,
    "total_tokens": 42,
    "completion_tokens": 29,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null,
  "kv_transfer_params": null
}

2. Data Formats

There are two common fine-tuning data formats: alpaca and sharegpt.

2.1 alpaca

Suited to single-turn question answering

Single-turn question answering

1
2
3
4
5
{
  "instruction": "介绍一下你自己",
  "input": "",
  "output": "我是 Ops Copilot，一个面向运维领域的智能助手，可以帮助你解决各种运维相关的问题。"
}

Multi-turn conversation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
[
  {
    "instruction": "你能帮我监控 Kubernetes 吗？",
    "input": "",
    "output": "当然可以，我可以通过 Prometheus 采集 Kubernetes 的监控指标，并提供告警与可视化。",
    "system": "你是 Ops Copilot，一个面向运维领域的智能助手。",
    "history": [
      [
        "你好",
        "你好，我是 Ops Copilot，一个面向运维领域的智能助手，可以帮助你解决各种运维相关的问题。"
      ]
    ]
  }
]

2.2 sharegpt

Suited to multi-turn conversation

Basic structure

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
{
  "conversations": [
    {
      "from": "system",
      "value": "你是一个专业的Python编程助手，请提供清晰、准确的代码示例。"
    },
    {
      "from": "human",
      "value": "如何用Python读取CSV文件？"
    },
    {
      "from": "gpt",
      "value": "可以使用pandas库来读取CSV文件：\n\n```python\nimport pandas as pd\ndf = pd.read_csv('file.csv')\nprint(df.head())\n```\n\n这样就能轻松加载和查看CSV数据了。"
    }
  ]
}

3. Fine-Tuning the Model

Start the environment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
nerdctl run -it \
        --security-opt apparmor=unconfined \
        --security-opt seccomp=unconfined \
        --gpus all \
        --ipc=host \
        --ulimit memlock=-1 \
        --ulimit stack=67108864 \
        --name llamafactory \
        --volume /data/models:/data/models \
        --entrypoint /bin/bash \
        hiyouga/llamafactory:0.9.4

The llamafactory-related tests below are all based on this environment.

Prepare the data

1
2
3
4
5
6
7
echo '[
  {
    "instruction": "介绍一下你自己",
    "input": "",
    "output": "我是 Ops Copilot，一个面向运维领域的智能助手，可以帮助你解决各种运维相关的问题。"
  }
]' > /data/models/dataset/alpaca_test.json

1
2
3
4
5
echo '{
  "alpaca_test.json": {
    "file_name": "alpaca_test.json"
  }
}' > /data/models/dataset/dataset_info.json

Run the fine-tuning

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
export CUDA_VISIBLE_DEVICES=0
llamafactory-cli train \
    --do_train \
    --stage sft \
    --model_name_or_path /data/models/Qwen3-0.6B \
    --dataset alpaca_test.json \
    --dataset_dir /data/models/dataset \
    --template qwen3 \
    --finetuning_type lora \
    --output_dir /data/models/Qwen3-0.6B-lora-sft \
    --per_device_train_batch_size 1 \
    --max_steps 20 \
    --learning_rate 1e-4 \
    --logging_steps 1 \
    --save_steps 10 \
    --save_total_limit 1 \
    --overwrite_output_dir \
    --warmup_ratio 0

1
2
3
4
5
6
7
***** train metrics *****
  epoch                    =       20.0
  total_flos               =     1991GF
  train_loss               =     1.0216
  train_runtime            = 0:00:05.97
  train_samples_per_second =      3.345
  train_steps_per_second   =      3.345

Inspect the lora weights

1
2
3
4
5
6
ls /data/models/Qwen3-0.6B-lora-sft

README.md                  all_results.json     special_tokens_map.json  trainer_log.jsonl
adapter_config.json        chat_template.jinja  tokenizer.json           trainer_state.json
adapter_model.safetensors  checkpoint-1         tokenizer_config.json    training_args.bin
added_tokens.json          merges.txt           train_results.json       vocab.json

Merge the weights

1
2
3
4
5
6
7
8
export CUDA_VISIBLE_DEVICES=0
llamafactory-cli export \
    --model_name_or_path /data/models/Qwen3-0.6B \
    --adapter_name_or_path /data/models/Qwen3-0.6B-lora-sft \
    --export_dir /data/models/Qwen3-0.6B-lora-merged \
    --template qwen \
    --export_size 2 \
    --finetuning_type lora

Inspect the merged model

1
2
3
4
5
ls /data/models/Qwen3-0.6B-lora-merged/

Modelfile            config.json             model.safetensors        tokenizer_config.json
added_tokens.json    generation_config.json  special_tokens_map.json  vocab.json
chat_template.jinja  merges.txt              tokenizer.json

Before and after fine-tuning, the model size does not change.

4. Testing the Fine-Tuned Model

4.1 Merging the lora Model

Start the service

1
2
3
4
5
6
export CUDA_VISIBLE_DEVICES=0
python3 -m vllm.entrypoints.openai.api_server \
  --model /data/models/Qwen3-0.6B-lora-merged \
  --served-model-name /data/models/Qwen3-0.6B \
  --host 0.0.0.0 \
  --port 8000

The merged model has better inference performance and is also easier to manage and deploy.

Test

1
2
3
4
5
6
7
8
curl -X POST "http://127.0.0.1:8000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "/data/models/Qwen3-0.6B",
    "messages": [
      {"role": "user", "content": "介绍一下你自己/no_think"}
    ]
  }'

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
{
  "id": "chatcmpl-9da628e2e9ed466e82b2c5ff2ea295ad",
  "object": "chat.completion",
  "created": 1758596914,
  "model": "/data/models/Qwen3-0.6B",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "<think>\n\n</think>\n\n我是 Ops Copilot，一个面向运维领域的智能助手，可以帮助你解决各种运维相关的问题。",
        "refusal": null,
        "annotations": null,
        "audio": null,
        "function_call": null,
        "tool_calls": [],
        "reasoning_content": null
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "prompt_tokens": 13,
    "total_tokens": 38,
    "completion_tokens": 25,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null,
  "kv_transfer_params": null
}

Seeing that the model’s self-introduction is Ops Copilot shows that the data we constructed has been trained into the model.

4.2 Loading the lora Model Separately

Start the service

1
2
3
4
5
6
7
8
export CUDA_VISIBLE_DEVICES=0
python3 -m vllm.entrypoints.openai.api_server \
  --model /data/models/Qwen3-0.6B \
  --served-model-name /data/models/Qwen3-0.6B \
  --enable-lora \
  --lora-modules ops-lora=/data/models/Qwen3-0.6B-lora-sft \
  --host 0.0.0.0 \
  --port 8000

Loading the lora model separately is more flexible and makes the model easier to share, and you can also load multiple lora models at the same time.

Test

Note that the model name here is the name of the lora module, ops-lora

1
2
3
4
5
6
7
8
curl -X POST "http://127.0.0.1:8000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ops-lora",
    "messages": [
      {"role": "user", "content": "介绍一下你自己/no_think"}
    ]
  }'

1
{"id":"chatcmpl-8ad998bdcbc3447892ba0228d7b5f55d","object":"chat.completion","created":1758597178,"model":"ops-lora","choices":[{"index":0,"message":{"role":"assistant","content":"<think>\n\n</think>\n\n我是 Ops Copilot，一个面向运维领域的智能助手，可以帮助你解决各种运维相关的问题。","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":13,"total_tokens":38,"completion_tokens":25,"prompt_tokens_details":null},"prompt_logprobs":null,"kv_transfer_params":null}

Merging the lora model and loading the lora model separately produce the same output.

5. Summary

This post mainly records the process of fine-tuning the Qwen3 model with LLaMAFactory, and tests the fine-tuned model. Many of the parameters were not studied in depth; the goal was just to walk through the fine-tuning process once, in preparation for fine-tuning the Ops Copilot model.

Today’s LLM toolchains are becoming more and more mature, and the related operations are becoming simpler and simpler. LLaMAFactory also provides a Web interface, which makes fine-tuning very convenient.