Django Application Performance Testing Based on Docker

1. Basic Website Performance Metrics

Concurrency: the number of sessions the server handles per unit of time
Throughput: the number of requests the server handles per unit of time
Response time: the total elapsed time from when the user sends a request to when the response is fully received, made up of network transmission time, service processing time, and other parts
Average response time: the average of the response times of all requests

2. Testing Tools

Performance testing has two levels: one is stress testing a single endpoint, and the other is stress testing in a production environment by simulating user operations. The former can be automated against CGIs with tools, while the latter usually requires tests customized to the business logic.

The main free stress testing tools include apache ab, webench, wrk, Gatling, sniper, hey, Siege, http_load, vegeta, t50, GoReplay, tcpcopy, gryphon, locust.io, and Jmeter. Several of them are described below.

2.1 ab

ab stands for: apache bench

Command format: ab [options] [http://]hostname[:port]/path

Main parameters:

1
2
  -n, total number of requests
  -c, number of concurrent connections

For example, generate 1000 requests at a concurrency of 50.

1
ab -c 50 -n 1000 http://127.0.0.1/

2.2 http_load

Before testing with http_load, you need to write the URLs to be tested into a file and pass that file to http_load as an argument.

Command format: http_load [-checksum] [-throttle] [-proxy host:port] [-verbose] [-timeout secs] [-sip sip_file] -parallel N | -rate N [-jitter] -fetches N | -seconds N url_file

Main parameters:
```bash`
-parallel, number of concurrent connections
-fetches, total number of requests

For example, generate 1000 requests at a concurrency of 50.

```bash
# http_load -parallel 50 -fetches 1000 url.txt

2.3 webbench, siege

webbench does not support specifying a total number of visits; it supports concurrency and test duration, and does not directly support HTTPS testing.

Command format: webbench [option]… URL

Main parameters:

1
2
-c, number of concurrent connections
-t, test duration

For example, test continuously for 20 seconds at a concurrency of 50

1
webbench -c 50 -t 20 http://127.0.0.1/

siege’s parameters and command format are similar to webbench’s.

1
siege -c 50 -t 20 http://127.0.0.1/

2.4 wrk

wrk uses some OS-specific high-performance I/O mechanisms, such as select, epoll, kqueue, etc., and can stress out a large amount of concurrency with very few threads.

Command format: wrk

Main parameters:

1
2
3
4
5
6
7
-c, number of connections established and kept with the server
-d, test duration
-t, how many threads to use for the test
-s, path to the Lua script
-H, add an HTTP header to every HTTP request
--latency, print latency statistics after the test ends
--timeout, timeout

For example, start 10 threads and run the test at a concurrency of 100 for 30 seconds.

1
wrk -t10 -c100 -d30s http://127.0.0.1

Compared with other tools, wrk can use Lua scripts to stress test more complex scenarios. For example, to test a POST request, you only need to prepare a Lua script.

post.lua

1
2
3
wrk.method = "POST"
wrk.headers["Content-Type"] = "application/x-www-form-urlencoded"
wrk.body = "key1=val1&key2=val2 "

Use the -s parameter

1
wrk -t10 -c100 -d30s  -s post.lua http://127.0.0.1

3. Test Demo

The deployment architecture of the Django application uses an Nginx container as a reverse proxy; behind it, requests are forwarded to Django through uWSGI, and Django accesses services such as MySQL and Redis, then returns the response along the original path.

Hardware: a laptop with an Intel i5-5300 2.3GHz, 8GB of memory, and a SAMSUNG SSD

Software: CentOS 7.2, Nginx with 2 workers, uWSGI with 8 workers, no gevent.

Below is a simple single test run:

Without DB access:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
wrk -t10 -c10000 -d10 'http://127.0.0.1'

Running 10s test @ http://127.0.0.1
  10 threads and 10000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   176.93ms  334.40ms   1.99s    90.73%
    Req/Sec   322.89    162.38   840.00     67.78%
  18351 requests in 10.06s, 7.70MB read
  Socket errors: connect 8989, read 56, write 0, timeout 549
Requests/sec:   1824.76
Transfer/sec:    784.08KB

With DB access:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
wrk -t10 -c10000 -d10 'http://127.0.0.1/books_fbv/'

Running 10s test @ http://127.0.0.1/books_fbv/
  10 threads and 10000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   393.67ms  274.98ms   1.99s    92.02%
    Req/Sec    69.44     70.51   360.00     82.23%
  3623 requests in 10.05s, 2.27MB read
  Socket errors: connect 8989, read 102, write 0, timeout 179
Requests/sec:    360.40
Transfer/sec:    231.58KB

Nginx forwarding static files:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
wrk -t10 -c10000 -d10 'http://127.0.0.1/static/index.html'

Running 10s test @ http://127.0.0.1/static/index.html
  10 threads and 10000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    26.33ms   54.99ms   1.34s    99.19%
    Req/Sec     4.84k     3.39k   10.72k    50.58%
  371148 requests in 10.08s, 89.54MB read
  Socket errors: connect 8989, read 0, write 0, timeout 19
Requests/sec:  36809.92
Transfer/sec:      8.88MB

Django forwarding static files:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
wrk -t10 -c10000 -d10 'http://127.0.0.1/static/index.html'

Running 10s test @ http://127.0.0.1/static/index.html
  10 threads and 10000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   268.50ms  282.75ms   1.99s    92.15%
    Req/Sec   120.43    107.12   414.00     53.24%
  6184 requests in 10.06s, 15.33MB read
  Socket errors: connect 8989, read 301, write 0, timeout 264
  Non-2xx or 3xx responses: 6184
Requests/sec:    615.00
Transfer/sec:      1.52MB

4. References

http://66note.com/wp/79