Skip to content

[aiohttp] - use batch updates #9932

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 9, 2025
Merged

[aiohttp] - use batch updates #9932

merged 1 commit into from
Jun 9, 2025

Conversation

@Reskov
Copy link
Contributor

@Reskov Reskov commented Jun 6, 2025

Before

  108652 requests in 5.10s, 21.71MB read
Requests/sec:  21305.19
Transfer/sec:      4.26MB
---------------------------------------------------------
 Running Warmup update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/2"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/2
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.11ms    1.57ms  77.90ms   97.59%
    Req/Sec     5.14k     0.96k    7.81k    75.06%
  Latency Distribution
     50%    0.90ms
     75%    1.21ms
     90%    1.58ms
     99%    5.24ms
  920212 requests in 30.02s, 183.90MB read
Requests/sec:  30650.80
Transfer/sec:      6.13MB
---------------------------------------------------------
 Queries: 1 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/1"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/1
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     0.94ms    1.00ms  49.10ms   97.21%
    Req/Sec     5.74k   603.68     7.74k    81.78%
  Latency Distribution
     50%  814.00us
     75%    1.05ms
     90%    1.34ms
     99%    3.43ms
  1027556 requests in 30.01s, 174.21MB read
Requests/sec:  34245.70
Transfer/sec:      5.81MB
STARTTIME 1749226060
ENDTIME 1749226090
---------------------------------------------------------
 Queries: 5 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/5"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/5
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.18ms  756.04us  27.04ms   90.42%
    Req/Sec     4.43k     0.91k    8.39k    67.65%
  Latency Distribution
     50%    1.04ms
     75%    1.40ms
     90%    1.82ms
     99%    3.31ms
  793474 requests in 30.10s, 231.48MB read
Requests/sec:  26361.55
Transfer/sec:      7.69MB
STARTTIME 1749226092
ENDTIME 1749226122
---------------------------------------------------------
 Queries: 10 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/10"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/10
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.74ms    1.73ms  55.19ms   96.37%
    Req/Sec     3.19k   580.80     5.19k    75.11%
  Latency Distribution
     50%    1.46ms
     75%    1.90ms
     90%    2.47ms
     99%    8.30ms
  571438 requests in 30.02s, 253.29MB read
Requests/sec:  19035.11
Transfer/sec:      8.44MB
STARTTIME 1749226124
ENDTIME 1749226154
---------------------------------------------------------
 Queries: 15 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/15"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/15
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.85ms  768.02us  33.57ms   82.68%
    Req/Sec     2.75k   226.78     6.98k    81.52%
  Latency Distribution
     50%    1.78ms
     75%    2.14ms
     90%    2.55ms
     99%    4.24ms
  492291 requests in 30.10s, 292.81MB read
Requests/sec:  16355.73
Transfer/sec:      9.73MB
STARTTIME 1749226156
ENDTIME 1749226187
---------------------------------------------------------
 Queries: 20 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/20"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/20
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.25ms    1.01ms  32.37ms   82.13%
    Req/Sec     2.27k   138.21     2.75k    78.94%
  Latency Distribution
     50%    2.13ms
     75%    2.63ms
     90%    3.21ms
     99%    5.27ms
  406830 requests in 30.01s, 303.63MB read
Requests/sec:  13556.05
Transfer/sec:     10.12MB
STARTTIME 1749226189
ENDTIME 1749226219

After

BENCHMARKING UPDATE ... 
---------------------------------------------------------
 Running Primer update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 5 -c 8 --timeout 8 -t 8 "http://tfb-server:8080/updates/2"
---------------------------------------------------------
Running 5s test @ http://tfb-server:8080/updates/2
  8 threads and 8 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   357.52us  228.20us   8.01ms   94.07%
    Req/Sec     2.91k     1.01k    5.74k    81.48%
  Latency Distribution
     50%  330.00us
     75%  433.00us
     90%  530.00us
     99%    1.06ms
  117338 requests in 5.10s, 23.45MB read
Requests/sec:  23008.80
Transfer/sec:      4.60MB
---------------------------------------------------------
 Running Warmup update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/2"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/2
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.61ms    3.24ms 102.07ms   96.18%
    Req/Sec     4.25k     0.98k    6.37k    75.33%
  Latency Distribution
     50%    1.04ms
     75%    1.45ms
     90%    2.11ms
     99%   14.07ms
  760865 requests in 30.01s, 152.06MB read
Requests/sec:  25354.35
Transfer/sec:      5.07MB
---------------------------------------------------------
 Queries: 1 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/1"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/1
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.41ms    2.92ms  87.74ms   97.78%
    Req/Sec     4.50k     0.92k    7.88k    79.11%
  Latency Distribution
     50%    1.01ms
     75%    1.42ms
     90%    1.94ms
     99%    9.35ms
  806344 requests in 30.03s, 136.71MB read
Requests/sec:  26852.62
Transfer/sec:      4.55MB
STARTTIME 1749225759
ENDTIME 1749225789
---------------------------------------------------------
 Queries: 5 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/5"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/5
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.48ms    2.56ms 149.46ms   97.90%
    Req/Sec     3.92k   660.05     6.63k    80.59%
  Latency Distribution
     50%    1.17ms
     75%    1.56ms
     90%    2.08ms
     99%    7.22ms
  703083 requests in 30.10s, 205.11MB read
Requests/sec:  23358.84
Transfer/sec:      6.81MB
STARTTIME 1749225791
ENDTIME 1749225821
---------------------------------------------------------
 Queries: 10 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/10"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/10
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.71ms    2.21ms  72.19ms   96.86%
    Req/Sec     3.36k     0.85k    5.11k    66.33%
  Latency Distribution
     50%    1.29ms
     75%    1.87ms
     90%    2.66ms
     99%    8.78ms
  602204 requests in 30.02s, 266.93MB read
Requests/sec:  20062.77
Transfer/sec:      8.89MB
STARTTIME 1749225823
ENDTIME 1749225853
---------------------------------------------------------
 Queries: 15 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/15"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/15
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.72ms    1.15ms  69.09ms   94.39%
    Req/Sec     3.03k   404.38     6.70k    72.97%
  Latency Distribution
     50%    1.57ms
     75%    1.95ms
     90%    2.42ms
     99%    5.07ms
  542657 requests in 30.10s, 322.77MB read
Requests/sec:  18028.96
Transfer/sec:     10.72MB
STARTTIME 1749225855
ENDTIME 1749225885
---------------------------------------------------------
 Queries: 20 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/20"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/20
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.00ms    1.19ms  70.75ms   92.90%
    Req/Sec     2.59k   285.76     5.81k    72.86%
  Latency Distribution
     50%    1.86ms
     75%    2.28ms
     90%    2.76ms
     99%    5.31ms
  464507 requests in 30.09s, 346.67MB read
Requests/sec:  15434.90
Transfer/sec:     11.52MB
STARTTIME 1749225887
ENDTIME 1749225917

united approach

BENCHMARKING UPDATE ... 
---------------------------------------------------------
 Running Primer update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 5 -c 8 --timeout 8 -t 8 "http://tfb-server:8080/updates/2"
---------------------------------------------------------
Running 5s test @ http://tfb-server:8080/updates/2
  8 threads and 8 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   466.78us  289.82us   9.50ms   97.08%
    Req/Sec     2.23k   192.62     2.71k    65.11%
  Latency Distribution
     50%  430.00us
     75%  509.00us
     90%  603.00us
     99%    1.18ms
  90192 requests in 5.10s, 18.02MB read
Requests/sec:  17690.22
Transfer/sec:      3.54MB
---------------------------------------------------------
 Running Warmup update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/2"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/2
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.06ms    6.76ms 157.20ms   96.41%
    Req/Sec     4.91k     1.04k    8.49k    71.78%
  Latency Distribution
     50%    0.90ms
     75%    1.27ms
     90%    1.89ms
     99%   32.43ms
  879943 requests in 30.01s, 175.86MB read
Requests/sec:  29320.87
Transfer/sec:      5.86MB
---------------------------------------------------------
 Queries: 1 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/1"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/1
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.68ms   12.52ms 281.32ms   97.04%
    Req/Sec     5.43k     1.11k   15.10k    71.35%
  Latency Distribution
     50%  801.00us
     75%    1.14ms
     90%    1.73ms
     99%   54.99ms
  972633 requests in 30.10s, 164.90MB read
Requests/sec:  32314.07
Transfer/sec:      5.48MB
STARTTIME 1749227557
ENDTIME 1749227587
---------------------------------------------------------
 Queries: 5 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/5"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/5
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.16ms  672.73us  23.25ms   88.52%
    Req/Sec     4.47k   528.92    10.77k    75.19%
  Latency Distribution
     50%    1.07ms
     75%    1.37ms
     90%    1.70ms
     99%    2.92ms
  801649 requests in 30.10s, 233.86MB read
Requests/sec:  26633.53
Transfer/sec:      7.77MB
STARTTIME 1749227589
ENDTIME 1749227619
---------------------------------------------------------
 Queries: 10 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/10"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/10
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.50ms    2.18ms  90.30ms   98.70%
    Req/Sec     3.69k   518.76     4.80k    65.06%
  Latency Distribution
     50%    1.27ms
     75%    1.66ms
     90%    2.09ms
     99%    4.53ms
  660978 requests in 30.00s, 292.98MB read
Requests/sec:  22030.63
Transfer/sec:      9.77MB
STARTTIME 1749227621
ENDTIME 1749227651
---------------------------------------------------------
 Queries: 15 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/15"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/15
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.62ms    0.97ms  51.23ms   94.73%
    Req/Sec     3.19k   225.37     3.93k    71.50%
  Latency Distribution
     50%    1.52ms
     75%    1.84ms
     90%    2.19ms
     99%    3.59ms
  570872 requests in 30.01s, 339.55MB read
Requests/sec:  19024.85
Transfer/sec:     11.32MB
STARTTIME 1749227653
ENDTIME 1749227683
---------------------------------------------------------
 Queries: 20 for update
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 30 -c 32 --timeout 8 -t 6 "http://tfb-server:8080/updates/20"
---------------------------------------------------------
Running 30s test @ http://tfb-server:8080/updates/20
  6 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.49ms    3.78ms 104.52ms   96.66%
    Req/Sec     2.48k   460.41     3.29k    77.78%
  Latency Distribution
     50%    1.87ms
     75%    2.35ms
     90%    3.10ms
     99%   19.78ms
  444861 requests in 30.02s, 332.01MB read
Requests/sec:  14820.45
Transfer/sec:     11.06MB

@Reskov Reskov marked this pull request as draft June 6, 2025 16:41
@Reskov Reskov mentioned this pull request Jun 6, 2025
Merged
@Reskov
Copy link
Contributor Author

Reskov commented Jun 6, 2025

also [] is slightly faster than tuple

tuple(zip(update_ids, numbers)): 0.764148 seconds
[*(zip(update_ids, numbers))]: 0.721439 seconds
Difference: 0.042709 seconds
Method 2 is faster by 5.92%

from random import sample, randint
import timeit

# Define number of queries
num_queries = 20

update_ids = sample(range(1, 10001), num_queries)
update_ids.sort()

numbers = [randint(1, 10000) for _ in range(num_queries)]
fetch_params = [(i,) for i in update_ids]

# Benchmark the two approaches
def method1():
    return tuple(zip(update_ids, numbers))

def method2():
    return [*(zip(update_ids, numbers))]

# Run benchmark
iterations = 1000000
time1 = timeit.timeit(method1, number=iterations)
time2 = timeit.timeit(method2, number=iterations)

print(f"tuple(zip(update_ids, numbers)): {time1:.6f} seconds")
print(f"[*(zip(update_ids, numbers))]: {time2:.6f} seconds")
print(f"Difference: {abs(time1-time2):.6f} seconds")
print(f"Method {'1' if time1 < time2 else '2'} is faster by {(max(time1,time2)/min(time1,time2)-1)*100:.2f}%")

@Reskov
Copy link
Contributor Author

Reskov commented Jun 6, 2025

@Dreamsorcerer please take a look 🙏

@Reskov Reskov marked this pull request as ready for review June 7, 2025 02:15
@msmith-techempower msmith-techempower merged commit d6fc514 into TechEmpower:master Jun 9, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@msmith-techempower msmith-techempower msmith-techempower approved these changes

+1 more reviewer

@Dreamsorcerer Dreamsorcerer Dreamsorcerer approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Reskov @Dreamsorcerer @msmith-techempower