Performance tests using "ab"

[jduan00]

Deployment

Apps and Postgres are deployed on fly.io with base configuration

• 1 shared cpu
• 256mb ram (app), 512mb ram (db)

Client: California, USA
Server: Singapore

Test command

ab -c 50 -n 200 -H "Authorization: Bearer xxxx" "https://api.eg1.io/v0/whereis/fdx-880121646970" 

Results

1. 50 concurrent requests
2. ~40 request per sec
3. ~22ms per request

Server Software:        Fly/2611eb36
Server Hostname:        api.eg1.io
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-ECDSA-CHACHA20-POLY1305,256,256
Server Temp Key:        ECDH X25519 253 bits
TLS Server Name:        api.eg1.io

Document Path:          /v0/whereis/fdx-880121646970
Document Length:        8086 bytes

Concurrency Level:      50
Time taken for tests:   11.470 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      4165500 bytes
HTML transferred:       4043000 bytes
Requests per second:    43.59 [#/sec] (mean)
Time per request:       1146.969 [ms] (mean)
Time per request:       22.939 [ms] (mean, across all concurrent requests)
Transfer rate:          354.66 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       60   92  29.0     83     339
Processing:   585  872 138.0    844    1797
Waiting:      585  870 138.2    841    1791
Total:        671  964 153.4    932    1962

Percentage of the requests served within a certain time (ms)
  50%    932
  66%    967
  75%    994
  80%   1004
  90%   1090
  95%   1312
  98%   1523
  99%   1623
 100%   1962 (longest request)

[jduan00]

ab -c 50 -n 500 https://api.eg1.io/v0/status/sfex-SF3148685401369

Results

1. 50 concurrent requests
2. ~80 request per sec
3. ~12ms per request

Server Software:        Fly/2611eb36
Server Hostname:        api.eg1.io
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-ECDSA-CHACHA20-POLY1305,256,256
Server Temp Key:        ECDH X25519 253 bits
TLS Server Name:        api.eg1.io

Document Path:          /v0/status/sfex-SF3148685401369
Document Length:        62 bytes

Concurrency Level:      50
Time taken for tests:   5.895 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      152500 bytes
HTML transferred:       31000 bytes
Requests per second:    84.81 [#/sec] (mean)
Time per request:       589.545 [ms] (mean)
Time per request:       11.791 [ms] (mean, across all concurrent requests)
Transfer rate:          25.26 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       64  119  24.1    120     304
Processing:   227  399  20.9    396     540
Waiting:      227  398  19.3    395     540
Total:        358  518  34.0    517     707

Percentage of the requests served within a certain time (ms)
  50%    517
  66%    530
  75%    538
  80%    544
  90%    558
  95%    575
  98%    582
  99%    601
 100%    707 (longest request)

[jduan00]

[jduan00]

New setup in SIN

1. 2x app servers (1cpu@256M)
2. 3x pg servers (1cpu@512M)

Results

100 concurrent clients

• Requests per second: 178.65
• Time per request: 5.598 [ms]

200 concurrent clients

• Requests per second: 271.73 [#/sec] (mean)
• Time per request: 3.680 [ms]

300 concurrent clients

Setup

1. 2 app node, 1 pg db node
2. With br compression and TCP keep-alive

• Requests per second: 419.59 [#/sec] (mean)
• Time per request: 2.383 [ms]

Testing details

ab -c 100 -n 1000 -H "Authorization: Bearer xxx" "https://api.eg1.io/v0/whereis/fdx-880121646970"

This is ApacheBench, Version 2.3 <$Revision: 1913912 $>
...

Server Software:        Fly/e660f5c79
Server Hostname:        api.eg1.io
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-ECDSA-CHACHA20-POLY1305,256,256
Server Temp Key:        ECDH X25519 253 bits
TLS Server Name:        api.eg1.io

Document Path:          /v0/whereis/fdx-880121646970
Document Length:        8187 bytes

Concurrency Level:      100
Time taken for tests:   5.598 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      8448000 bytes
HTML transferred:       8187000 bytes
Requests per second:    178.65 [#/sec] (mean)
Time per request:       559.751 [ms] (mean)
Time per request:       5.598 [ms] (mean, across all concurrent requests)
Transfer rate:          1473.87 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       58   99  45.2     81     328
Processing:   209  395  27.0    384     531
Waiting:      204  389  24.2    379     531
Total:        416  493  62.1    468     811

Percentage of the requests served within a certain time (ms)
  50%    468
  66%    491
  75%    506
  80%    517
  90%    593
  95%    640
  98%    681
  99%    704
 100%    811 (longest request)

Concurrency Level:      200
Time taken for tests:   3.680 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      8448000 bytes
HTML transferred:       8187000 bytes
Requests per second:    271.73 [#/sec] (mean)
Time per request:       736.013 [ms] (mean)
Time per request:       3.680 [ms] (mean, across all concurrent requests)
Transfer rate:          2241.81 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       69  162  81.0    123     344
Processing:   372  421  45.8    407     702
Waiting:      370  414  42.6    397     645
Total:        451  584 114.2    532     885

Percentage of the requests served within a certain time (ms)
  50%    532
  66%    581
  75%    627
  80%    687
  90%    781
  95%    852
  98%    867
  99%    872
 100%    885 (longest request)

ab -k -c300 -n2000 -H "Authorization: Bearer xxx" \
   -H "Accept-Encoding: br" \
   "https://api.eg1.io/v0/whereis/fdx-880820328666"

Server Software:        Fly/1e6fdd9a8
Server Hostname:        api.eg1.io
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-ECDSA-CHACHA20-POLY1305,256,256
Server Temp Key:        ECDH X25519 253 bits
TLS Server Name:        api.eg1.io

Document Path:          /v0/whereis/fdx-880820328666
Document Length:        933 bytes

Concurrency Level:      300
Time taken for tests:   4.767 seconds
Complete requests:      2000
Failed requests:        0
Keep-Alive requests:    0
Total transferred:      2430000 bytes
HTML transferred:       1866000 bytes
Requests per second:    419.59 [#/sec] (mean)
Time per request:       714.985 [ms] (mean)
Time per request:       2.383 [ms] (mean, across all concurrent requests)
Transfer rate:          497.85 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       75  167 109.2    130    1137
Processing:   371  428  66.4    411    1518
Waiting:      371  426  62.7    409    1337
Total:        460  596 160.5    543    1696

Percentage of the requests served within a certain time (ms)
  50%    543
  66%    562
  75%    578
  80%    589
  90%    894
  95%   1000
  98%   1097
  99%   1173
 100%   1696 (longest request)

[jduan00]

A lot of work with #105