Perf: testing with "ab" benchmark and optimzation

[jduan00]

Summary of Performance Test Results

Test Setup

• Server: fly.io (Singapore, except for fly.io SJC, and local Mac M4), Postgres + app
• Client: California, USA
• Tool: ApacheBench (ab)
• Test Endpoint: /v0/whereis/fdx-880121646970

Performance Summary

In our tests, the application sustained about 500–600 requests per second on Fly.io. Beyond this point, adding more CPU cores or memory didn’t increase throughput — it mainly improved stability under heavy load. The main performance constraints were network latency between our test location and the servers, along with limits in the cloud platforms.

After enabling multi‑core support in Deno, we observed a ~2x performance boost on local machines with very low latency - 2 core over 1 core is only 30% faster. However, in cloud environments this change had little impact because performance remained network‑bound rather than CPU‑bound.

On IBM cloud with 8 core (6 core for Deno) and increase pg connection limit to 1000, we are getting 2639 rps.

In ideal conditions — such as running locally on a Mac — the application can process well over 3,000 requests per second, demonstrating that the codebase is highly efficient and that server location and network speed are the key factors affecting real‑world results.

---

Requests Per Second (RPS) Comparison by Environment

Environment	Configuration	Requests Per Second (RPS)
Fly.io Singapore (SG)	1 CPU / 512MB × 2 VMs	446
Fly.io San Jose (SJC)	1 CPU / 512MB × 2 VMs	520
Fly.io San Jose (SJC)	2 CPU / 1024MB × 2 VMs	551
IBM Cloud Local	1 CPU / 4GB	693
IBM Cloud Local	6 CPU / 8GB	2,639
Mac M4 Local	1 CPU / 1GB	2,531
Mac M4 Local	4 CPU / 4GB	5,498

---

Requests per Second (rps) Breakdown

VM Configuration	RAM per VM	# VMs	Concurrency	rps	Time/req (ms)
1 shared CPU	256MB	1	50	43	22.939
1 shared CPU	256MB	1	100	App crashed
2 CPUs	1GB	2	50	171
1 CPU	512MB	2	50	177
1 CPU	512MB	2	100	~181
1 CPU	512MB	2	200	~323
1 CPU	512MB	2	250	~371
1 CPU	512MB	2	500	~446	2.242
1 CPU (fly.io SJC)	512MB	2	500	~520	1.920
2 CPU (fly.io SJC)	512MB	2	500	App crashed
2 CPU (fly.io SJC)	1024MB	2	500	551	1.814
4 CPU (fly.io SJC)	8192MB	2	500	510	1.960
1 CPU (IBM VPC, local)	4GB	1	500	693	1.441
2 CPU (IBM VPC, local)	4GB	1	500	665	1.503
6 CPU (IBM VPC, local)	4GB	1	500	2639	0.379
1 CPU (Mac M4, local)	1GB	1	500	2531	0.342
2 CPU (Mac M4, local)	1GB	1	500	3346	0.299
4 CPU (Mac M4, local)	4GB	1	500	5499	0.182

Additional Notes

• Out-of-memory errors occurred at high concurrency with low RAM (256MB).
• No failed requests in higher-concurrency tests, but rps plateaued at higher values.
• Memory usage stabilized around 300MB (with 2 VMs, 512MB each).
• To further improve performance, use Deno workers (for multi-core) Perf: Use deno serve --parallel to utilize multiple cores #190 and increase horizontal scaling.

Make the following change to docker-compose.yaml to increase pg connection limit.

services:
  pg-whereis:
    image: postgres
    command: postgres -c max_connections=1000
    container_name: pg-whereis

Per request process time and optimization

• On Mac M4 local, it is ~4ms (app code + pg process)
• To explore using KV cache to lower this to 1ms

Discussed in #41

[jduan00]

2 VM, each 1 core x 512MB: 177 rps

% fly scale vm shared-cpu-1x --memory=512

% ab -n 2000 -c 100 -H "Authorization: Bearer xxx" "https://api.eg1.io/v0/whereis/fdx-880121646970"

Requests per second:    177.56 [#/sec] (mean)
Time per request:       563.199 [ms] (mean)
Time per request:       5.632 [ms] (mean, across all concurrent requests)
Transfer rate:          1324.92 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       55   88  31.9     77     223
Processing:   243  436  35.8    448     547
Waiting:      241  432  34.5    444     533
Total:        318  524  53.8    525     720

Percentage of the requests served within a certain time (ms)
  50%    525
  66%    535
  75%    540
  80%    546
  90%    598
  95%    642
  98%    672
  99%    683
 100%    720 (longest request)

[jduan00]

Findings

1. Each VM to use 512MB at least. The best we can get is 177 rps. Performance is not CPU bound. Memory utilization is about 300MB.
2. Deno can only use one core, so we need to use Deno workers to take advantage of multiple cores.
3. We need to find out how to horizontally scale within Fly.io.

[jduan00]

Increase concurrent request to 200 => 316 rps

ab -n 2000 -c 200 https://api.eg1.io/v0/status/fdx-880121646970

Requests per second:    316.54 [#/sec] (mean)
Time per request:       631.823 [ms] (mean)
Time per request:       3.159 [ms] (mean, across all concurrent requests)
Transfer rate:          153.02 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       58  119  70.1     92     360
Processing:   227  434  32.5    446     598
Waiting:      227  433  32.4    445     598
Total:        357  553  78.4    538     824

Percentage of the requests served within a certain time (ms)
  50%    538
  66%    551
  75%    564
  80%    574
  90%    673
  95%    752
  98%    782
  99%    807
 100%    824 (longest request)

---

Increase concurrent request to 250 => 373 rps

ab -n 2000 -c 250 https://api.eg1.io/v0/status/fdx-880121646970

Requests per second:    373.35 [#/sec] (mean)
Time per request:       669.613 [ms] (mean)
Time per request:       2.678 [ms] (mean, across all concurrent requests)
Transfer rate:          180.48 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       62  135  91.5     96    1113
Processing:   243  435  33.3    449     522
Waiting:      243  434  33.2    448     516
Total:        330  569 102.7    544    1568

---

Increase concurrent request to 500 => 446 rps

Concurrency Level:      500
Time taken for tests:   4.484 seconds
Complete requests:      2000
Failed requests:        0
Total transferred:      15282000 bytes
HTML transferred:       14660000 bytes
Requests per second:    446.05 [#/sec] (mean)
Time per request:       1120.957 [ms] (mean)
Time per request:       2.242 [ms] (mean, across all concurrent requests)
Transfer rate:          3328.37 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       64  351 292.2    252    1841
Processing:   386  492  66.9    494    1184
Waiting:      376  443  36.2    452     653
Total:        518  843 303.2    738    2648

[jduan00]

optimize Deno in multiple core VMs

https://www.perplexity.ai/search/3a72338e-318f-46e5-9b7c-4e10d8240a4d

Deno 2.0 introduced the  deno serve --parallel  command, which creates multi-threaded web servers automatically

[jduan00]

fly.io deployment in SJC, and with Cloudflare proxy

ab -n 5000 -c200 https://api.eagl.io/v0/status/fdx-012301230123

Server Software:        cloudflare
Server Hostname:        api.eagl.io
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128
Server Temp Key:        ECDH X25519 253 bits
TLS Server Name:        api.eagl.io

Requests per second:    520.72 [#/sec] (mean)
Time per request:       384.083 [ms] (mean)
Time per request:       1.920 [ms] (mean, across all concurrent requests)
Transfer rate:          361.38 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       66  285 209.3    238    2243
Processing:    39   84  39.7     78    1818
Waiting:       38   78  37.1     72    1817
Total:        141  369 214.3    322    3005

Percentage of the requests served within a certain time (ms)
  50%    322
  66%    349
  75%    368
  80%    379
  90%    423
  95%    716
  98%   1317
  99%   1363
 100%   3005 (longest request)

Local Macbook Air (M4) running docker containers for app and postgres db

ab -n 10000 -c200 localhost:8080/v0/status/fdx-012301230123

  Requests per second:    2924.75 [#/sec] (mean)
  Time per request:       68.382 [ms] (mean)
  Time per request:       0.342 [ms] (mean, across all concurrent requests)
  Transfer rate:          1059.65 [Kbytes/sec] received

  Connection Times (ms)
                min  mean[+/-sd] median   max
  Connect:        0    0   0.5      0      14
  Processing:     9   67  16.0     65     188
  Waiting:        3   67  16.0     65     188
  Total:          9   68  15.9     65     188

  Percentage of the requests served within a certain time (ms)
    50%     65
    66%     71
    75%     75
    80%     78
    90%     88
    95%     97
    98%    109
    99%    118
   100%    188 (longest request)

[samshdn]

For Postgres-related latency, see #208