Using VLLM Benchmark for Model Performance Testing
· ☕ 4 min read
VLLM Benchmark is a tool provided by VLLM for testing model performance, supporting a variety of inference backends. This article mainly records some of the process of using VLLM Benchmark to test model performance.
1. Starting the Model Service 1 2 3 4 5 6 7 8 9 10 11 12 13 14 python -m vllm.