Skip to content

Perf experiments #682

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 11 commits into
base: main
Choose a base branch
from
Draft

Perf experiments #682

wants to merge 11 commits into from

Conversation

ardatan
Copy link
Member

@ardatan ardatan commented May 13, 2025

No description provided.

Copy link

github-actions bot commented May 13, 2025

Overview for: federation/constant-vus-subgraphs-delay

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 300 VUs over 30s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
cosmo 173 5360 total, 0 failed avg: 893ms, p95: 2149ms
grafbase 166 5148 total, 0 failed avg: 911ms, p95: 2555ms
apollo-router 157 4892 total, 0 failed avg: 876ms, p95: 2272ms
hive-gateway-bun 92 3009 total, 0 failed avg: 3005ms, p95: 4679ms
hive-gateway 91 3038 total, 0 failed avg: 3074ms, p95: 14820ms
apollo-server 85 2848 total, 0 failed avg: 3290ms, p95: 28280ms
mercurius 75 2429 total, 0 failed avg: 3819ms, p95: 5481ms
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 16020      ✗ 0    
     data_received..................: 470 MB  15 MB/s
     data_sent......................: 6.4 MB  206 kB/s
     http_req_blocked...............: avg=2.88ms   min=1.69µs  med=4.03µs   max=2.18s p(90)=6.63µs  p(95)=193.52µs
     http_req_connecting............: avg=2.25ms   min=0s      med=0s       max=2.18s p(90)=0s      p(95)=124.9µs 
     http_req_duration..............: avg=893.28ms min=3.33ms  med=776.74ms max=4.5s  p(90)=1.86s   p(95)=2.14s   
       { expected_response:true }...: avg=893.28ms min=3.33ms  med=776.74ms max=4.5s  p(90)=1.86s   p(95)=2.14s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 5360 
     http_req_receiving.............: avg=268ms    min=34.37µs med=104.5µs  max=3.83s p(90)=1.17s   p(95)=1.55s   
     http_req_sending...............: avg=21.55ms  min=8.31µs  med=21.25µs  max=2.21s p(90)=196.2µs p(95)=10.98ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s    p(90)=0s      p(95)=0s      
     http_req_waiting...............: avg=603.72ms min=3.22ms  med=590.79ms max=2.58s p(90)=1.04s   p(95)=1.16s   
     http_reqs......................: 5360    173.755449/s
     iteration_duration.............: avg=1.68s    min=24.21ms med=1.47s    max=7.3s  p(90)=3.2s    p(95)=3.82s   
     iterations.....................: 5340    173.107107/s
     vus............................: 300     min=300      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 15384      ✗ 0    
     data_received..................: 452 MB  15 MB/s
     data_sent......................: 6.1 MB  197 kB/s
     http_req_blocked...............: avg=1.06ms   min=1.46µs  med=3.71µs   max=634.94ms p(90)=6.22µs   p(95)=6.91ms 
     http_req_connecting............: avg=909.07µs min=0s      med=0s       max=525.39ms p(90)=0s       p(95)=6.72ms 
     http_req_duration..............: avg=910.76ms min=3.13ms  med=692.08ms max=6.73s    p(90)=2.07s    p(95)=2.55s  
       { expected_response:true }...: avg=910.76ms min=3.13ms  med=692.08ms max=6.73s    p(90)=2.07s    p(95)=2.55s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 5148 
     http_req_receiving.............: avg=356.19ms min=34.52µs med=88.6µs   max=6s       p(90)=1.58s    p(95)=1.98s  
     http_req_sending...............: avg=15.28ms  min=8.25µs  med=17.02µs  max=2.46s    p(90)=429.69µs p(95)=12.61ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=539.28ms min=3.03ms  med=501.81ms max=2.58s    p(90)=981.74ms p(95)=1.23s  
     http_reqs......................: 5148    166.246703/s
     iteration_duration.............: avg=1.77s    min=22.92ms med=1.5s     max=12.35s   p(90)=3.62s    p(95)=4.51s  
     iterations.....................: 5128    165.600834/s
     vus............................: 300     min=300      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 14616      ✗ 0    
     data_received..................: 429 MB  14 MB/s
     data_sent......................: 5.8 MB  187 kB/s
     http_req_blocked...............: avg=3.52ms   min=1.71µs  med=3.54µs   max=2.25s p(90)=6.76µs   p(95)=3.9ms  
     http_req_connecting............: avg=2.96ms   min=0s      med=0s       max=2.25s p(90)=0s       p(95)=3.65ms 
     http_req_duration..............: avg=875.79ms min=7.34ms  med=701.57ms max=4.94s p(90)=1.95s    p(95)=2.27s  
       { expected_response:true }...: avg=875.79ms min=7.34ms  med=701.57ms max=4.94s p(90)=1.95s    p(95)=2.27s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 4892 
     http_req_receiving.............: avg=333.5ms  min=31.88µs med=97.24µs  max=4.27s p(90)=1.27s    p(95)=1.73s  
     http_req_sending...............: avg=16.98ms  min=8.59µs  med=17.17µs  max=3.13s p(90)=458.74µs p(95)=11.33ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s    p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=525.3ms  min=7.16ms  med=484.54ms max=2.71s p(90)=937.84ms p(95)=1.14s  
     http_reqs......................: 4892    157.596041/s
     iteration_duration.............: avg=1.86s    min=27.39ms med=1.57s    max=9.6s  p(90)=3.72s    p(95)=4.38s  
     iterations.....................: 4872    156.95174/s
     vus............................: 39      min=39       max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8967      ✗ 0    
     data_received..................: 264 MB  8.1 MB/s
     data_sent......................: 3.6 MB  109 kB/s
     http_req_blocked...............: avg=358.9µs  min=1.87µs  med=3.51µs  max=26.92ms  p(90)=153.03µs p(95)=3.12ms  
     http_req_connecting............: avg=314.87µs min=0s      med=0s      max=15.18ms  p(90)=13.96µs  p(95)=2.93ms  
     http_req_duration..............: avg=3s       min=16.09ms med=2.75s   max=6.73s    p(90)=4.37s    p(95)=4.67s   
       { expected_response:true }...: avg=3s       min=16.09ms med=2.75s   max=6.73s    p(90)=4.37s    p(95)=4.67s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 3009 
     http_req_receiving.............: avg=67.3ms   min=40.11µs med=142.6µs max=2.31s    p(90)=151.32ms p(95)=527.12ms
     http_req_sending...............: avg=1.66ms   min=9.83µs  med=18.3µs  max=698.24ms p(90)=264.33µs p(95)=2.04ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=2.93s    min=15.89ms med=2.72s   max=6.73s    p(90)=4.26s    p(95)=4.61s   
     http_reqs......................: 3009    92.200942/s
     iteration_duration.............: avg=3.12s    min=85.78ms med=2.8s    max=7.22s    p(90)=4.52s    p(95)=4.84s   
     iterations.....................: 2989    91.588108/s
     vus............................: 120     min=120     max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 9054      ✗ 0    
     data_received..................: 267 MB  8.0 MB/s
     data_sent......................: 3.6 MB  108 kB/s
     http_req_blocked...............: avg=1.22ms   min=1.82µs   med=4.22µs   max=36.17ms  p(90)=36.08µs  p(95)=13.16ms
     http_req_connecting............: avg=1.18ms   min=0s       med=0s       max=23.88ms  p(90)=0s       p(95)=12.94ms
     http_req_duration..............: avg=3.07s    min=13.74ms  med=1.67s    max=32.77s   p(90)=2.91s    p(95)=14.82s 
       { expected_response:true }...: avg=3.07s    min=13.74ms  med=1.67s    max=32.77s   p(90)=2.91s    p(95)=14.82s 
     http_req_failed................: 0.00%   ✓ 0         ✗ 3038 
     http_req_receiving.............: avg=1.49ms   min=46.54µs  med=103.34µs max=260.83ms p(90)=593.86µs p(95)=2.37ms 
     http_req_sending...............: avg=353.21µs min=10.75µs  med=24.02µs  max=69.83ms  p(90)=221.49µs p(95)=1.45ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=3.07s    min=13.59ms  med=1.67s    max=32.77s   p(90)=2.91s    p(95)=14.81s 
     http_reqs......................: 3038    91.318463/s
     iteration_duration.............: avg=3.13s    min=119.09ms med=1.72s    max=32.78s   p(90)=3.02s    p(95)=15.01s 
     iterations.....................: 3018    90.717288/s
     vus............................: 30      min=30      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-server`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8484      ✗ 0    
     data_received..................: 250 MB  7.5 MB/s
     data_sent......................: 3.4 MB  102 kB/s
     http_req_blocked...............: avg=516.99µs min=1.52µs   med=4.04µs   max=13.41ms  p(90)=176.54µs p(95)=4.68ms  
     http_req_connecting............: avg=497.63µs min=0s       med=0s       max=12.85ms  p(90)=133.01µs p(95)=4.58ms  
     http_req_duration..............: avg=3.28s    min=12ms     med=1.42s    max=32.78s   p(90)=1.99s    p(95)=28.28s  
       { expected_response:true }...: avg=3.28s    min=12ms     med=1.42s    max=32.78s   p(90)=1.99s    p(95)=28.28s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 2848 
     http_req_receiving.............: avg=328.12µs min=44.34µs  med=109.26µs max=289.08ms p(90)=209.49µs p(95)=310.79µs
     http_req_sending...............: avg=107.27µs min=9.13µs   med=22.13µs  max=12.03ms  p(90)=129.13µs p(95)=571.12µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.28s    min=11.84ms  med=1.42s    max=32.78s   p(90)=1.99s    p(95)=28.27s  
     http_reqs......................: 2848    85.754557/s
     iteration_duration.............: avg=3.32s    min=303.34ms med=1.44s    max=32.79s   p(90)=2.01s    p(95)=28.75s  
     iterations.....................: 2828    85.152348/s
     vus............................: 24      min=24      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7227      ✗ 0    
     data_received..................: 213 MB  6.6 MB/s
     data_sent......................: 2.9 MB  90 kB/s
     http_req_blocked...............: avg=997.41µs min=1.82µs   med=4.16µs  max=25.55ms  p(90)=2.67ms   p(95)=10.09ms
     http_req_connecting............: avg=968.62µs min=0s       med=0s      max=15.57ms  p(90)=2.42ms   p(95)=10ms   
     http_req_duration..............: avg=3.81s    min=12.49ms  med=3.8s    max=7.81s    p(90)=4.72s    p(95)=5.48s  
       { expected_response:true }...: avg=3.81s    min=12.49ms  med=3.8s    max=7.81s    p(90)=4.72s    p(95)=5.48s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 2429 
     http_req_receiving.............: avg=4.65ms   min=39.01µs  med=108.2µs max=564.93ms p(90)=284.05µs p(95)=821.6µs
     http_req_sending...............: avg=247.6µs  min=9.28µs   med=23.65µs max=14.71ms  p(90)=286.1µs  p(95)=2.94ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s       p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=3.81s    min=12.32ms  med=3.8s    max=7.81s    p(90)=4.72s    p(95)=5.47s  
     http_reqs......................: 2429    75.463219/s
     iteration_duration.............: avg=3.87s    min=445.15ms med=3.82s   max=7.82s    p(90)=4.73s    p(95)=5.51s  
     iterations.....................: 2409    74.841867/s
     vus............................: 47      min=47      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented May 13, 2025

Overview for: federation/constant-vus-over-time

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 300 VUs over 30s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
cosmo 176 5450 total, 0 failed avg: 829ms, p95: 2179ms
grafbase 169 5249 total, 0 failed avg: 912ms, p95: 2063ms
apollo-router 154 4779 total, 0 failed avg: 934ms, p95: 2216ms
apollo-server 90 3004 total, 0 failed avg: 3116ms, p95: 24816ms
hive-gateway-bun 90 2975 total, 0 failed avg: 3080ms, p95: 5403ms
hive-gateway 85 2853 total, 0 failed avg: 3274ms, p95: 17358ms
mercurius 77 2476 total, 0 failed avg: 3715ms, p95: 5324ms
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 16290      ✗ 0    
     data_received..................: 478 MB  16 MB/s
     data_sent......................: 6.5 MB  209 kB/s
     http_req_blocked...............: avg=1.29ms   min=1.51µs  med=3.83µs   max=1.21s    p(90)=6.66µs   p(95)=252.63µs
     http_req_connecting............: avg=883.67µs min=0s      med=0s       max=897.12ms p(90)=0s       p(95)=133.46µs
     http_req_duration..............: avg=829.27ms min=3.28ms  med=689.52ms max=4.84s    p(90)=1.74s    p(95)=2.17s   
       { expected_response:true }...: avg=829.27ms min=3.28ms  med=689.52ms max=4.84s    p(90)=1.74s    p(95)=2.17s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 5450 
     http_req_receiving.............: avg=304.59ms min=35.88µs med=104.22µs max=3.92s    p(90)=1.15s    p(95)=1.65s   
     http_req_sending...............: avg=16.8ms   min=8.61µs  med=18.67µs  max=2.28s    p(90)=150.65µs p(95)=7.07ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=507.86ms min=3.14ms  med=478ms    max=2.04s    p(90)=960.42ms p(95)=1.05s   
     http_reqs......................: 5450    176.318741/s
     iteration_duration.............: avg=1.66s    min=19.26ms med=1.37s    max=8.71s    p(90)=3.43s    p(95)=4.18s   
     iterations.....................: 5430    175.6717/s
     vus............................: 300     min=300      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 15687      ✗ 0    
     data_received..................: 461 MB  15 MB/s
     data_sent......................: 6.2 MB  201 kB/s
     http_req_blocked...............: avg=1.68ms   min=1.54µs  med=3.19µs   max=1.04s p(90)=5.34µs   p(95)=1.45ms 
     http_req_connecting............: avg=1.38ms   min=0s      med=0s       max=1.04s p(90)=0s       p(95)=1.02ms 
     http_req_duration..............: avg=911.9ms  min=3.03ms  med=783.94ms max=5.7s  p(90)=1.65s    p(95)=2.06s  
       { expected_response:true }...: avg=911.9ms  min=3.03ms  med=783.94ms max=5.7s  p(90)=1.65s    p(95)=2.06s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 5249 
     http_req_receiving.............: avg=163.91ms min=34.17µs med=75.26µs  max=4.2s  p(90)=746.78ms p(95)=1.13s  
     http_req_sending...............: avg=21.63ms  min=8.21µs  med=14.69µs  max=4.93s p(90)=412.47µs p(95)=13.42ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s    p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=726.34ms min=2.96ms  med=692.15ms max=2.49s p(90)=1.17s    p(95)=1.39s  
     http_reqs......................: 5249    169.68652/s
     iteration_duration.............: avg=1.74s    min=30.5ms  med=1.55s    max=7.16s p(90)=3.21s    p(95)=3.82s  
     iterations.....................: 5229    169.039972/s
     vus............................: 300     min=300      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 14277      ✗ 0    
     data_received..................: 419 MB  14 MB/s
     data_sent......................: 5.7 MB  183 kB/s
     http_req_blocked...............: avg=3.97ms   min=1.77µs  med=3.73µs   max=2.23s p(90)=6.74µs p(95)=27.11ms
     http_req_connecting............: avg=3.85ms   min=0s      med=0s       max=2.23s p(90)=0s     p(95)=26.37ms
     http_req_duration..............: avg=933.94ms min=6.99ms  med=826.31ms max=5.15s p(90)=1.87s  p(95)=2.21s  
       { expected_response:true }...: avg=933.94ms min=6.99ms  med=826.31ms max=5.15s p(90)=1.87s  p(95)=2.21s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 4779 
     http_req_receiving.............: avg=349.25ms min=35.24µs med=104.83µs max=4.73s p(90)=1.26s  p(95)=1.53s  
     http_req_sending...............: avg=22.28ms  min=8.06µs  med=19.18µs  max=1.87s p(90)=8.48ms p(95)=32.39ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s    p(90)=0s     p(95)=0s     
     http_req_waiting...............: avg=562.4ms  min=6.86ms  med=524.63ms max=2.37s p(90)=1.09s  p(95)=1.22s  
     http_reqs......................: 4779    154.213351/s
     iteration_duration.............: avg=1.9s     min=30.64ms med=1.66s    max=8.72s p(90)=3.95s  p(95)=4.62s  
     iterations.....................: 4759    153.567971/s
     vus............................: 8       min=8        max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-server`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8952      ✗ 0    
     data_received..................: 264 MB  8.0 MB/s
     data_sent......................: 3.6 MB  108 kB/s
     http_req_blocked...............: avg=559.43µs min=1.3µs    med=3.11µs  max=19.58ms p(90)=107.16µs p(95)=2.71ms  
     http_req_connecting............: avg=528.98µs min=0s       med=0s      max=19.54ms p(90)=48.27µs  p(95)=2.35ms  
     http_req_duration..............: avg=3.11s    min=10.35ms  med=1.35s   max=32.62s  p(90)=2.12s    p(95)=24.81s  
       { expected_response:true }...: avg=3.11s    min=10.35ms  med=1.35s   max=32.62s  p(90)=2.12s    p(95)=24.81s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 3004 
     http_req_receiving.............: avg=240.34µs min=41.33µs  med=99.49µs max=176.9ms p(90)=189.97µs p(95)=261.17µs
     http_req_sending...............: avg=113.47µs min=8.56µs   med=15.64µs max=22.65ms p(90)=72.95µs  p(95)=270.91µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.11s    min=10.27ms  med=1.35s   max=32.62s  p(90)=2.12s    p(95)=24.81s  
     http_reqs......................: 3004    90.967128/s
     iteration_duration.............: avg=3.15s    min=150.93ms med=1.37s   max=32.63s  p(90)=2.14s    p(95)=24.92s  
     iterations.....................: 2984    90.361488/s
     vus............................: 6       min=6       max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8865      ✗ 0    
     data_received..................: 261 MB  8.0 MB/s
     data_sent......................: 3.5 MB  108 kB/s
     http_req_blocked...............: avg=2.58ms min=1.77µs   med=4.38µs   max=96.14ms p(90)=977.74µs p(95)=26.47ms 
     http_req_connecting............: avg=2.48ms min=0s       med=0s       max=49.56ms p(90)=241.29µs p(95)=26.05ms 
     http_req_duration..............: avg=3.07s  min=16.85ms  med=2.79s    max=6.67s   p(90)=4.65s    p(95)=5.4s    
       { expected_response:true }...: avg=3.07s  min=16.85ms  med=2.79s    max=6.67s   p(90)=4.65s    p(95)=5.4s    
     http_req_failed................: 0.00%   ✓ 0         ✗ 2975 
     http_req_receiving.............: avg=62ms   min=39.6µs   med=133.01µs max=1.6s    p(90)=17.14ms  p(95)=553.53ms
     http_req_sending...............: avg=1.38ms min=9.63µs   med=23.77µs  max=696.9ms p(90)=418.03µs p(95)=2.08ms  
     http_req_tls_handshaking.......: avg=0s     min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.01s  min=16.69ms  med=2.74s    max=6.67s   p(90)=4.63s    p(95)=5.39s   
     http_reqs......................: 2975    90.793094/s
     iteration_duration.............: avg=3.16s  min=223.77ms med=2.85s    max=6.97s   p(90)=4.73s    p(95)=5.42s   
     iterations.....................: 2955    90.18272/s
     vus............................: 125     min=125     max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8499      ✗ 0    
     data_received..................: 251 MB  7.5 MB/s
     data_sent......................: 3.4 MB  102 kB/s
     http_req_blocked...............: avg=426.33µs min=2.15µs   med=5.28µs   max=21.99ms  p(90)=239.88µs p(95)=4.29ms
     http_req_connecting............: avg=381.47µs min=0s       med=0s       max=9.38ms   p(90)=123.7µs  p(95)=3.88ms
     http_req_duration..............: avg=3.27s    min=13.29ms  med=1.73s    max=32.78s   p(90)=3.3s     p(95)=17.35s
       { expected_response:true }...: avg=3.27s    min=13.29ms  med=1.73s    max=32.78s   p(90)=3.3s     p(95)=17.35s
     http_req_failed................: 0.00%   ✓ 0         ✗ 2853 
     http_req_receiving.............: avg=1.03ms   min=46.22µs  med=117.26µs max=129.66ms p(90)=611.56µs p(95)=2.09ms
     http_req_sending...............: avg=366.19µs min=11.46µs  med=30.81µs  max=96.1ms   p(90)=259.21µs p(95)=1.24ms
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s    
     http_req_waiting...............: avg=3.27s    min=13.16ms  med=1.73s    max=32.78s   p(90)=3.3s     p(95)=17.35s
     http_reqs......................: 2853    85.835791/s
     iteration_duration.............: avg=3.33s    min=181.26ms med=1.76s    max=32.79s   p(90)=3.33s    p(95)=18.24s
     iterations.....................: 2833    85.234068/s
     vus............................: 23      min=23      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7368      ✗ 0    
     data_received..................: 217 MB  6.8 MB/s
     data_sent......................: 2.9 MB  92 kB/s
     http_req_blocked...............: avg=752.06µs min=1.63µs   med=3.65µs  max=18.29ms  p(90)=1.26ms   p(95)=6.8ms   
     http_req_connecting............: avg=735.21µs min=0s       med=0s      max=18.24ms  p(90)=1.09ms   p(95)=6.67ms  
     http_req_duration..............: avg=3.71s    min=11.46ms  med=3.68s   max=7.95s    p(90)=4.68s    p(95)=5.32s   
       { expected_response:true }...: avg=3.71s    min=11.46ms  med=3.68s   max=7.95s    p(90)=4.68s    p(95)=5.32s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 2476 
     http_req_receiving.............: avg=1.82ms   min=37.44µs  med=95.91µs max=468.37ms p(90)=231.68µs p(95)=436.82µs
     http_req_sending...............: avg=137.75µs min=8.62µs   med=20.62µs max=11.26ms  p(90)=165.83µs p(95)=1.17ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.71s    min=11.31ms  med=3.68s   max=7.95s    p(90)=4.68s    p(95)=5.32s   
     http_reqs......................: 2476    77.503755/s
     iteration_duration.............: avg=3.78s    min=412.63ms med=3.71s   max=8s       p(90)=4.7s     p(95)=5.37s   
     iterations.....................: 2456    76.877715/s
     vus............................: 230     min=230     max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented May 13, 2025

Overview for: federation/constant-vus-subgraphs-delay-resources

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 500 VUs over 30s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
cosmo 175 5500 total, 0 failed avg: 1551ms, p95: 3716ms
grafbase 161 5074 total, 0 failed avg: 1367ms, p95: 3453ms
apollo-router 157 4946 total, 11 failed avg: 1520ms, p95: 3293ms ❌ 11 failed requests, 11 non-200 responses, 11 unexpected GraphQL errors, non-compatible response structure (1)
hive-gateway 91 3220 total, 0 failed avg: 4952ms, p95: 32806ms
apollo-server 90 3157 total, 0 failed avg: 5094ms, p95: 33076ms
hive-gateway-bun 86 3053 total, 0 failed avg: 5249ms, p95: 8939ms
mercurius 73 2437 total, 0 failed avg: 6389ms, p95: 11079ms
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 16440      ✗ 0    
     data_received..................: 483 MB  15 MB/s
     data_sent......................: 6.5 MB  208 kB/s
     http_req_blocked...............: avg=5.04ms   min=1.5µs   med=3.45µs   max=6.64s  p(90)=21.7µs   p(95)=16.66ms
     http_req_connecting............: avg=3.05ms   min=0s      med=0s       max=3.12s  p(90)=0s       p(95)=16.36ms
     http_req_duration..............: avg=1.55s    min=3.26ms  med=1.3s     max=8.64s  p(90)=3.36s    p(95)=3.71s  
       { expected_response:true }...: avg=1.55s    min=3.26ms  med=1.3s     max=8.64s  p(90)=3.36s    p(95)=3.71s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 5500 
     http_req_receiving.............: avg=567.85ms min=37.22µs med=100µs    max=7.34s  p(90)=2.11s    p(95)=2.91s  
     http_req_sending...............: avg=35.67ms  min=8.02µs  med=15.52µs  max=5.63s  p(90)=712.76µs p(95)=3.75ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s     p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=947.95ms min=3.18ms  med=951.62ms max=3.65s  p(90)=1.64s    p(95)=1.91s  
     http_reqs......................: 5500    175.041376/s
     iteration_duration.............: avg=2.79s    min=20.67ms med=2.41s    max=12.03s p(90)=5.64s    p(95)=6.88s  
     iterations.....................: 5480    174.404861/s
     vus............................: 175     min=175      max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 15162      ✗ 0    
     data_received..................: 446 MB  14 MB/s
     data_sent......................: 6.0 MB  192 kB/s
     http_req_blocked...............: avg=4.36ms   min=1.63µs  med=4.32µs   max=4.23s  p(90)=254.76µs p(95)=16.89ms
     http_req_connecting............: avg=3.93ms   min=0s      med=0s       max=4.23s  p(90)=80.52µs  p(95)=16.49ms
     http_req_duration..............: avg=1.36s    min=3.36ms  med=1.12s    max=6.69s  p(90)=2.94s    p(95)=3.45s  
       { expected_response:true }...: avg=1.36s    min=3.36ms  med=1.12s    max=6.69s  p(90)=2.94s    p(95)=3.45s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 5074 
     http_req_receiving.............: avg=316.41ms min=33.47µs med=86.71µs  max=4.26s  p(90)=1.28s    p(95)=2.55s  
     http_req_sending...............: avg=20.37ms  min=8.13µs  med=21.05µs  max=4.2s   p(90)=953µs    p(95)=7.87ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s     p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=1.03s    min=3.27ms  med=948.71ms max=3.69s  p(90)=1.82s    p(95)=2.44s  
     http_reqs......................: 5074    161.312424/s
     iteration_duration.............: avg=2.99s    min=32.75ms med=2.68s    max=11.79s p(90)=5.64s    p(95)=6.78s  
     iterations.....................: 5054    160.676585/s
     vus............................: 241     min=241      max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✗ response code was 200
      ↳  99% — ✓ 4915 / ✗ 11
     ✗ no graphql errors
      ↳  99% — ✓ 4915 / ✗ 11
     ✗ valid response structure
      ↳  99% — ✓ 4915 / ✗ 1

     █ setup

     checks.........................: 99.84% ✓ 14745      ✗ 23   
     data_received..................: 433 MB 14 MB/s
     data_sent......................: 5.9 MB 187 kB/s
     http_req_blocked...............: avg=11.27ms  min=1.6µs   med=3.58µs  max=4.54s p(90)=791.62µs p(95)=19.56ms
     http_req_connecting............: avg=10.17ms  min=0s      med=0s      max=3.5s  p(90)=520.15µs p(95)=19.35ms
     http_req_duration..............: avg=1.51s    min=6.73ms  med=1.33s   max=6.78s p(90)=2.83s    p(95)=3.29s  
       { expected_response:true }...: avg=1.51s    min=6.73ms  med=1.33s   max=6.78s p(90)=2.83s    p(95)=3.29s  
     http_req_failed................: 0.22%  ✓ 11         ✗ 4935 
     http_req_receiving.............: avg=360.64ms min=0s      med=94.66µs max=5.68s p(90)=1.22s    p(95)=1.93s  
     http_req_sending...............: avg=33.34ms  min=8.03µs  med=16.99µs max=4.86s p(90)=1.73ms   p(95)=13.77ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s    p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=1.12s    min=6.66ms  med=1.08s   max=5.43s p(90)=2.05s    p(95)=2.27s  
     http_reqs......................: 4946   157.771946/s
     iteration_duration.............: avg=3.06s    min=27.87ms med=2.7s    max=14.5s p(90)=5.7s     p(95)=6.76s  
     iterations.....................: 4926   157.133968/s
     vus............................: 208    min=208      max=500
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 9600      ✗ 0    
     data_received..................: 283 MB  8.0 MB/s
     data_sent......................: 3.8 MB  109 kB/s
     http_req_blocked...............: avg=3.07ms   min=2.13µs   med=4.78µs   max=48.96ms  p(90)=18.33ms  p(95)=22.56ms
     http_req_connecting............: avg=3ms      min=0s       med=0s       max=41.85ms  p(90)=18.21ms  p(95)=22.39ms
     http_req_duration..............: avg=4.95s    min=13.91ms  med=1.75s    max=34.69s   p(90)=19.19s   p(95)=32.8s  
       { expected_response:true }...: avg=4.95s    min=13.91ms  med=1.75s    max=34.69s   p(90)=19.19s   p(95)=32.8s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 3220 
     http_req_receiving.............: avg=3.55ms   min=43.35µs  med=109.15µs max=429.9ms  p(90)=673.42µs p(95)=3.83ms 
     http_req_sending...............: avg=823.46µs min=10.12µs  med=27.09µs  max=144.55ms p(90)=1.24ms   p(95)=2.77ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=4.94s    min=13.79ms  med=1.75s    max=34.69s   p(90)=19.19s   p(95)=32.8s  
     http_reqs......................: 3220    91.535271/s
     iteration_duration.............: avg=5.03s    min=169.48ms med=1.79s    max=34.7s    p(90)=19.56s   p(95)=32.9s  
     iterations.....................: 3200    90.966729/s
     vus............................: 22      min=22      max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-server`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 9411      ✗ 0    
     data_received..................: 277 MB  7.9 MB/s
     data_sent......................: 3.7 MB  107 kB/s
     http_req_blocked...............: avg=2.73ms   min=1.55µs   med=3.09µs  max=41.54ms p(90)=15.16ms  p(95)=20.81ms 
     http_req_connecting............: avg=2.65ms   min=0s       med=0s      max=41.5ms  p(90)=14.9ms   p(95)=20.36ms 
     http_req_duration..............: avg=5.09s    min=10.05ms  med=1.47s   max=34.64s  p(90)=31.54s   p(95)=33.07s  
       { expected_response:true }...: avg=5.09s    min=10.05ms  med=1.47s   max=34.64s  p(90)=31.54s   p(95)=33.07s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 3157 
     http_req_receiving.............: avg=205.27µs min=38.8µs   med=99.99µs max=27.91ms p(90)=220.85µs p(95)=316.42µs
     http_req_sending...............: avg=236.87µs min=8.08µs   med=16.18µs max=49.77ms p(90)=558.89µs p(95)=986.76µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=5.09s    min=9.96ms   med=1.47s   max=34.64s  p(90)=31.54s   p(95)=33.07s  
     http_reqs......................: 3157    90.015173/s
     iteration_duration.............: avg=5.14s    min=287.14ms med=1.49s   max=34.65s  p(90)=31.58s   p(95)=33.11s  
     iterations.....................: 3137    89.444915/s
     vus............................: 15      min=15      max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 9099      ✗ 0    
     data_received..................: 268 MB  7.6 MB/s
     data_sent......................: 3.6 MB  103 kB/s
     http_req_blocked...............: avg=3.31ms   min=1.98µs   med=4.03µs   max=48.44ms  p(90)=18.29ms  p(95)=26.15ms 
     http_req_connecting............: avg=3.23ms   min=0s       med=0s       max=48.41ms  p(90)=17.98ms  p(95)=25.94ms 
     http_req_duration..............: avg=5.24s    min=15.96ms  med=4.98s    max=12.07s   p(90)=8.23s    p(95)=8.93s   
       { expected_response:true }...: avg=5.24s    min=15.96ms  med=4.98s    max=12.07s   p(90)=8.23s    p(95)=8.93s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 3053 
     http_req_receiving.............: avg=109.92ms min=41.57µs  med=241.83µs max=3.6s     p(90)=181.05ms p(95)=974.37ms
     http_req_sending...............: avg=1.69ms   min=10.21µs  med=21.93µs  max=401.73ms p(90)=1.51ms   p(95)=7.73ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=5.13s    min=15.45ms  med=4.88s    max=12.07s   p(90)=8.12s    p(95)=8.93s   
     http_reqs......................: 3053    86.891128/s
     iteration_duration.............: avg=5.36s    min=226.86ms med=5.05s    max=12.18s   p(90)=8.28s    p(95)=8.97s   
     iterations.....................: 3033    86.32191/s
     vus............................: 35      min=35      max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7251      ✗ 0    
     data_received..................: 214 MB  6.5 MB/s
     data_sent......................: 2.9 MB  88 kB/s
     http_req_blocked...............: avg=3.16ms  min=1.71µs   med=4.12µs   max=37.83ms  p(90)=17.26ms  p(95)=21.76ms 
     http_req_connecting............: avg=3.1ms   min=0s       med=0s       max=37.8ms   p(90)=16.98ms  p(95)=21.6ms  
     http_req_duration..............: avg=6.38s   min=11.3ms   med=6.35s    max=12.77s   p(90)=7.91s    p(95)=11.07s  
       { expected_response:true }...: avg=6.38s   min=11.3ms   med=6.35s    max=12.77s   p(90)=7.91s    p(95)=11.07s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 2437 
     http_req_receiving.............: avg=1.34ms  min=37.09µs  med=100.13µs max=410.69ms p(90)=263.54µs p(95)=483.08µs
     http_req_sending...............: avg=379.2µs min=9.63µs   med=25.25µs  max=15.97ms  p(90)=1.54ms   p(95)=2.9ms   
     http_req_tls_handshaking.......: avg=0s      min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=6.38s   min=11.2ms   med=6.35s    max=12.77s   p(90)=7.91s    p(95)=11.07s  
     http_reqs......................: 2437    73.917458/s
     iteration_duration.............: avg=6.5s    min=416.36ms med=6.38s    max=12.78s   p(90)=7.92s    p(95)=11.12s  
     iterations.....................: 2417    73.310832/s
     vus............................: 15      min=15      max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented May 13, 2025

Overview for: federation/ramping-vus

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was trying to reach 2000 concurrent VUs over 60s

Comparison

Comparison

Gateway duration(p95)⬇️ RPS Requests Durations Notes
grafbase 5924ms 170 12171 total, 0 failed avg: 2741ms, p95: 5925ms, max: 15645ms, med: 2533ms
cosmo 5944ms 176 12374 total, 0 failed avg: 2912ms, p95: 5945ms, max: 16490ms, med: 2580ms
apollo-router 8725ms 163 11477 total, 453 failed avg: 2814ms, p95: 8725ms, max: 19586ms, med: 1962ms ❌ 453 failed requests, 453 non-200 responses, 453 unexpected GraphQL errors
hive-gateway-bun 28686ms 94 7759 total, 0 failed avg: 11308ms, p95: 28686ms, max: 37995ms, med: 10245ms
mercurius 47026ms 44 4155 total, 596 failed avg: 19101ms, p95: 47027ms, max: 52473ms, med: 14933ms ❌ 596 failed requests, 590 non-200 responses, 590 unexpected GraphQL errors
hive-gateway 52794ms 92 8026 total, 0 failed avg: 11597ms, p95: 52795ms, max: 59452ms, med: 2687ms
apollo-server 59999ms 84 7841 total, 524 failed avg: 11883ms, p95: 60000ms, max: 60072ms, med: 2143ms ❌ 524 failed requests, 524 non-200 responses, 524 unexpected GraphQL errors
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 36453      ✗ 0     
     data_received..................: 1.1 GB  15 MB/s
     data_sent......................: 14 MB   202 kB/s
     http_req_blocked...............: avg=356.49ms min=1.5µs   med=3.53µs  max=10.67s p(90)=849.46ms p(95)=2.82s   
     http_req_connecting............: avg=353.52ms min=0s      med=0s      max=10.67s p(90)=837.65ms p(95)=2.78s   
     http_req_duration..............: avg=2.74s    min=3.14ms  med=2.53s   max=15.64s p(90)=5.28s    p(95)=5.92s   
       { expected_response:true }...: avg=2.74s    min=3.14ms  med=2.53s   max=15.64s p(90)=5.28s    p(95)=5.92s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 12171 
     http_req_receiving.............: avg=211.67ms min=33.52µs med=76.73µs max=14.17s p(90)=543.41ms p(95)=1.69s   
     http_req_sending...............: avg=139.01ms min=7.93µs  med=16.35µs max=8.73s  p(90)=306.27ms p(95)=907.22ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=2.39s    min=3.08ms  med=1.99s   max=10.36s p(90)=5.02s    p(95)=5.46s   
     http_reqs......................: 12171   170.040795/s
     iteration_duration.............: avg=6.18s    min=22.33ms med=5.29s   max=30.18s p(90)=12.23s   p(95)=14.72s  
     iterations.....................: 12151   169.761375/s
     vus............................: 600     min=70       max=1925
     vus_max........................: 2000    min=2000     max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 37062      ✗ 0     
     data_received..................: 1.1 GB  16 MB/s
     data_sent......................: 15 MB   209 kB/s
     http_req_blocked...............: avg=382.36ms min=1.48µs  med=3.47µs  max=10.4s  p(90)=1.06s    p(95)=3.2s    
     http_req_connecting............: avg=338.23ms min=0s      med=0s      max=10.4s  p(90)=862.78ms p(95)=3.15s   
     http_req_duration..............: avg=2.91s    min=3.31ms  med=2.57s   max=16.49s p(90)=5.39s    p(95)=5.94s   
       { expected_response:true }...: avg=2.91s    min=3.31ms  med=2.57s   max=16.49s p(90)=5.39s    p(95)=5.94s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 12374 
     http_req_receiving.............: avg=202.22ms min=34.67µs med=82.32µs max=11.21s p(90)=760.75ms p(95)=1.35s   
     http_req_sending...............: avg=119.87ms min=7.7µs   med=15.87µs max=11.31s p(90)=205.56ms p(95)=740.98ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=2.59s    min=3.12ms  med=1.82s   max=15.5s  p(90)=5.17s    p(95)=5.67s   
     http_reqs......................: 12374   176.424366/s
     iteration_duration.............: avg=5.78s    min=8.98ms  med=4.93s   max=25.94s p(90)=11.08s   p(95)=13.16s  
     iterations.....................: 12354   176.139212/s
     vus............................: 5       min=5        max=2000
     vus_max........................: 2000    min=2000     max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✗ response code was 200
      ↳  96% — ✓ 11004 / ✗ 453
     ✗ no graphql errors
      ↳  96% — ✓ 11004 / ✗ 453
     ✓ valid response structure

     █ setup

     checks.........................: 97.32% ✓ 33012      ✗ 906   
     data_received..................: 967 MB 14 MB/s
     data_sent......................: 14 MB  194 kB/s
     http_req_blocked...............: avg=669.86ms min=1.5µs   med=3.95µs  max=14.06s p(90)=2.44s    p(95)=4.27s 
     http_req_connecting............: avg=649.6ms  min=0s      med=0s      max=14.06s p(90)=2.38s    p(95)=4.18s 
     http_req_duration..............: avg=2.81s    min=6.73ms  med=1.96s   max=19.58s p(90)=6.67s    p(95)=8.72s 
       { expected_response:true }...: avg=2.79s    min=6.73ms  med=1.93s   max=19.58s p(90)=6.67s    p(95)=8.73s 
     http_req_failed................: 3.94%  ✓ 453        ✗ 11024 
     http_req_receiving.............: avg=740ms    min=0s      med=85.3µs  max=16.29s p(90)=3.22s    p(95)=5.73s 
     http_req_sending...............: avg=293.43ms min=8.26µs  med=21.37µs max=18.08s p(90)=499.52ms p(95)=1.11s 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s     p(90)=0s       p(95)=0s    
     http_req_waiting...............: avg=1.78s    min=6.64ms  med=1.43s   max=10.8s  p(90)=3.75s    p(95)=4.38s 
     http_reqs......................: 11477  163.345526/s
     iteration_duration.............: avg=6.36s    min=48.29ms med=4.82s   max=31.86s p(90)=13.95s   p(95)=16.21s
     iterations.....................: 11457  163.060877/s
     vus............................: 152    min=64       max=1999
     vus_max........................: 2000   min=2000     max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 23217     ✗ 0     
     data_received..................: 681 MB  8.3 MB/s
     data_sent......................: 9.2 MB  113 kB/s
     http_req_blocked...............: avg=5.92ms   min=1.65µs   med=3.88µs   max=445.25ms p(90)=2.86ms   p(95)=33.27ms 
     http_req_connecting............: avg=5.8ms    min=0s       med=0s       max=381.33ms p(90)=2.62ms   p(95)=32.29ms 
     http_req_duration..............: avg=11.3s    min=17.84ms  med=10.24s   max=37.99s   p(90)=24.21s   p(95)=28.68s  
       { expected_response:true }...: avg=11.3s    min=17.84ms  med=10.24s   max=37.99s   p(90)=24.21s   p(95)=28.68s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 7759  
     http_req_receiving.............: avg=110.78ms min=40.05µs  med=243.99µs max=7.63s    p(90)=3.94ms   p(95)=205.37ms
     http_req_sending...............: avg=2.73ms   min=9.39µs   med=24.15µs  max=477.62ms p(90)=507.91µs p(95)=12.36ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=11.19s   min=17.24ms  med=10.22s   max=37.4s    p(90)=24.2s    p(95)=28.55s  
     http_reqs......................: 7759    94.897608/s
     iteration_duration.............: avg=11.42s   min=210.76ms med=10.27s   max=38.48s   p(90)=24.37s   p(95)=28.79s  
     iterations.....................: 7739    94.652995/s
     vus............................: 15      min=15      max=2000
     vus_max........................: 2000    min=2000    max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `mercurius`

K6 Output

     ✗ response code was 200
      ↳  85% — ✓ 3539 / ✗ 590
     ✗ no graphql errors
      ↳  85% — ✓ 3539 / ✗ 590
     ✓ valid response structure

     █ setup

     checks.........................: 89.99% ✓ 10617     ✗ 1180  
     data_received..................: 312 MB 3.3 MB/s
     data_sent......................: 6.0 MB 64 kB/s
     http_req_blocked...............: avg=193.7µs  min=1.75µs   med=3.86µs  max=31.5ms   p(90)=474.14µs p(95)=600.24µs
     http_req_connecting............: avg=163.98µs min=0s       med=0s      max=31.4ms   p(90)=398.61µs p(95)=517.77µs
     http_req_duration..............: avg=19.1s    min=10.48ms  med=14.93s  max=52.47s   p(90)=43.69s   p(95)=47.02s  
       { expected_response:true }...: avg=15.18s   min=10.48ms  med=11.95s  max=44.37s   p(90)=32.02s   p(95)=43.42s  
     http_req_failed................: 14.34% ✓ 596       ✗ 3559  
     http_req_receiving.............: avg=5.93ms   min=0s       med=99.3µs  max=617.15ms p(90)=303.84µs p(95)=715.87µs
     http_req_sending...............: avg=53.04µs  min=9.4µs    med=23.03µs max=10.28ms  p(90)=63.5µs   p(95)=81.97µs 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=19.09s   min=10.41ms  med=14.93s  max=52.47s   p(90)=43.51s   p(95)=47.02s  
     http_reqs......................: 4155   44.422029/s
     iteration_duration.............: avg=19.17s   min=116.74ms med=14.96s  max=52.58s   p(90)=43.71s   p(95)=46.26s  
     iterations.....................: 4129   44.144057/s
     vus............................: 827    min=60      max=2000
     vus_max........................: 2000   min=2000    max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 24018     ✗ 0     
     data_received..................: 705 MB  8.1 MB/s
     data_sent......................: 9.5 MB  110 kB/s
     http_req_blocked...............: avg=4.04ms min=2.04µs  med=4.98µs   max=279.79ms p(90)=2.85ms   p(95)=23.04ms
     http_req_connecting............: avg=3.98ms min=0s      med=0s       max=279.54ms p(90)=2.6ms    p(95)=22.54ms
     http_req_duration..............: avg=11.59s min=12.33ms med=2.68s    max=59.45s   p(90)=45.21s   p(95)=52.79s 
       { expected_response:true }...: avg=11.59s min=12.33ms med=2.68s    max=59.45s   p(90)=45.21s   p(95)=52.79s 
     http_req_failed................: 0.00%   ✓ 0         ✗ 8026  
     http_req_receiving.............: avg=1.05ms min=44.45µs med=108.47µs max=475.66ms p(90)=524.75µs p(95)=1.47ms 
     http_req_sending...............: avg=1.89ms min=9.68µs  med=29.25µs  max=383.36ms p(90)=417.13µs p(95)=9.54ms 
     http_req_tls_handshaking.......: avg=0s     min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=11.59s min=11.99ms med=2.68s    max=59.44s   p(90)=45.21s   p(95)=52.79s 
     http_reqs......................: 8026    92.30115/s
     iteration_duration.............: avg=11.67s min=58.31ms med=2.72s    max=59.52s   p(90)=45.3s    p(95)=52.93s 
     iterations.....................: 8006    92.071145/s
     vus............................: 27      min=27      max=2000
     vus_max........................: 2000    min=2000    max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-server`

K6 Output

     ✗ response code was 200
      ↳  93% — ✓ 7297 / ✗ 524
     ✗ no graphql errors
      ↳  93% — ✓ 7297 / ✗ 524
     ✓ valid response structure

     █ setup

     checks.........................: 95.43% ✓ 21891     ✗ 1048  
     data_received..................: 643 MB 6.9 MB/s
     data_sent......................: 9.3 MB 101 kB/s
     http_req_blocked...............: avg=528.82µs min=1.65µs  med=3.97µs   max=93.51ms  p(90)=370.36µs p(95)=645.39µs
     http_req_connecting............: avg=498.71µs min=0s      med=0s       max=69.76ms  p(90)=295.65µs p(95)=546.03µs
     http_req_duration..............: avg=11.88s   min=10.21ms med=2.14s    max=1m0s     p(90)=55.56s   p(95)=59.99s  
       { expected_response:true }...: avg=8.43s    min=10.21ms med=2.07s    max=59.97s   p(90)=37.18s   p(95)=44.56s  
     http_req_failed................: 6.68%  ✓ 524       ✗ 7317  
     http_req_receiving.............: avg=307.71µs min=0s      med=105.99µs max=163.17ms p(90)=227.16µs p(95)=357.23µs
     http_req_sending...............: avg=212.83µs min=9.19µs  med=20.97µs  max=67.16ms  p(90)=64.11µs  p(95)=104.39µs
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=11.88s   min=10.11ms med=2.14s    max=1m0s     p(90)=55.56s   p(95)=59.99s  
     http_reqs......................: 7841   84.589607/s
     iteration_duration.............: avg=11.92s   min=59.6ms  med=2.16s    max=1m0s     p(90)=55.61s   p(95)=1m0s    
     iterations.....................: 7821   84.373845/s
     vus............................: 53     min=53      max=2000
     vus_max........................: 2000   min=2000    max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview

@ardatan ardatan force-pushed the try-perf-experiments branch from a3cde79 to 661a87f Compare May 13, 2025 17:09
@ardatan ardatan force-pushed the try-perf-experiments branch from 2daba2d to bf7b733 Compare May 15, 2025 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant