Skip to content

Commit ba59650

Browse files
committed
asim: increase range count in high_cpu_25nodes test
Previously, the CPU was not converging for the mma_only test, such that the standard deviation of CPU was 0.13, when it should be nearly 0. This was due to using too few ranges for the number of nodes in the cluster, so the ranges weren't being equally distributed across stores. This change increases the number of ranges from 50 to 500 which is sufficient for a more even distribution. The even distribution of ranges allows the CPU to similarly even out across the cluster stores, resulting in a new standard deviation of 0.01. The placement_type is now `replica_placement`, and the ranges are distributed across the first 9 nodes to make this test more realistic by keeping the CPU below 100%. As well, the initial workload is also removed since it did not add any value to the test scenario, and was a cause for confusion in understanding this test setup. Release note: none. Fixes #156844 Epic: CRDB-55052
1 parent 33ebee8 commit ba59650

File tree

1 file changed

+38
-41
lines changed

1 file changed

+38
-41
lines changed

pkg/kv/kvserver/asim/tests/testdata/non_rand/mma/high_cpu_25nodes.txt

Lines changed: 38 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -8,51 +8,48 @@ gen_cluster nodes=25 node_cpu_rate_capacity=8000000000
88
setting split_queue_enabled=false
99
----
1010

11-
# This workload will be initially evenly distributed over the cluster.
12-
gen_ranges ranges=50 min_key=0 max_key=10000
11+
# A workload is added over the entire keyspace, which is initially placed only
12+
# on s1-s9.
13+
gen_ranges ranges=500 min_key=0 max_key=20000 placement_type=replica_placement
14+
{s1,s2,s3}:1
15+
{s4,s5,s6}:1
16+
{s7,s8,s9}:1
1317
----
18+
{s1:*,s2,s3}:1
19+
{s4:*,s5,s6}:1
20+
{s7:*,s8,s9}:1
1421

15-
# TODO(tbg): likely accidentally too low.
16-
gen_load rate=15000 rw_ratio=0.95 min_block=100 max_block=100 request_cpu_per_access=100 raft_cpu_per_write=20 min_key=0 max_key=10000
22+
gen_load rate=15000 rw_ratio=0.95 min_block=1 max_block=1 request_cpu_per_access=700000 raft_cpu_per_write=200000 min_key=0 max_key=20000
1723
----
18-
0.00 access-vcpus, 0.00 raft-vcpus, 73 KiB/s goodput
24+
10.50 access-vcpus, 0.15 raft-vcpus, 750 B/s goodput
1925

20-
# Another workload is added over the second half of the keyspace, which is initially
21-
# only mainly on s1-s3 due to the skewed distribution.
22-
gen_ranges ranges=50 min_key=10001 max_key=20000 placement_type=skewed
26+
eval duration=25m samples=1 seed=42 cfgs=(mma-only,mma-count) metrics=(cpu,cpu_util,write_bytes_per_second,replicas,leases)
2327
----
24-
25-
gen_load rate=15000 rw_ratio=0.95 min_block=1 max_block=1 request_cpu_per_access=7000000 raft_cpu_per_write=20000 min_key=10001 max_key=20000
26-
----
27-
105.00 access-vcpus, 0.02 raft-vcpus, 750 B/s goodput
28-
29-
eval duration=20m samples=1 seed=42 cfgs=(mma-only,mma-count) metrics=(cpu,cpu_util,write_bytes_per_second,replicas,leases)
30-
----
31-
cpu#1: last: [s1=6305516202, s2=4203121714, s3=4209044827, s4=6316721595, s5=4197368968, s6=2107290360, s7=2099077812, s8=4207644575, s9=4199517611, s10=4195522966, s11=4205024256, s12=4198000753, s13=4202637211, s14=4186734543, s15=4199295119, s16=4199174047, s17=4204480499, s18=6290045651, s19=4193666150, s20=4211828495, s21=4201735135, s22=2096004946, s23=4201402387, s24=4193586357, s25=4193056496] (stddev=1029616922.87, mean=4200699947.00, sum=105017498675)
32-
cpu#1: thrash_pct: [s1=6%, s2=47%, s3=81%, s4=21%, s5=11%, s6=2%, s7=1%, s8=3%, s9=2%, s10=3%, s11=2%, s12=3%, s13=3%, s14=2%, s15=11%, s16=2%, s17=3%, s18=3%, s19=3%, s20=2%, s21=3%, s22=2%, s23=3%, s24=2%, s25=2%] (sum=222%)
33-
cpu_util#1: last: [s1=0.79, s2=0.53, s3=0.53, s4=0.79, s5=0.52, s6=0.26, s7=0.26, s8=0.53, s9=0.52, s10=0.52, s11=0.53, s12=0.52, s13=0.53, s14=0.52, s15=0.52, s16=0.52, s17=0.53, s18=0.79, s19=0.52, s20=0.53, s21=0.53, s22=0.26, s23=0.53, s24=0.52, s25=0.52] (stddev=0.13, mean=0.53, sum=13)
34-
cpu_util#1: thrash_pct: [s1=6%, s2=47%, s3=81%, s4=21%, s5=11%, s6=2%, s7=1%, s8=3%, s9=2%, s10=3%, s11=2%, s12=3%, s13=3%, s14=2%, s15=11%, s16=2%, s17=3%, s18=3%, s19=3%, s20=2%, s21=3%, s22=2%, s23=3%, s24=2%, s25=2%] (sum=222%)
35-
leases#1: first: [s1=29, s2=13, s3=4, s4=2, s5=2, s6=2, s7=3, s8=3, s9=2, s10=3, s11=4, s12=1, s13=2, s14=2, s15=4, s16=3, s17=3, s18=3, s19=2, s20=3, s21=3, s22=2, s23=2, s24=1, s25=2] (stddev=5.56, mean=4.00, sum=100)
36-
leases#1: last: [s1=5, s2=4, s3=4, s4=4, s5=4, s6=3, s7=4, s8=4, s9=4, s10=4, s11=5, s12=3, s13=3, s14=4, s15=5, s16=4, s17=4, s18=6, s19=4, s20=4, s21=4, s22=3, s23=4, s24=3, s25=4] (stddev=0.69, mean=4.00, sum=100)
37-
leases#1: thrash_pct: [s1=0%, s2=40%, s3=71%, s4=15%, s5=8%, s6=0%, s7=0%, s8=0%, s9=0%, s10=0%, s11=0%, s12=0%, s13=0%, s14=0%, s15=8%, s16=0%, s17=0%, s18=0%, s19=0%, s20=0%, s21=0%, s22=0%, s23=0%, s24=0%, s25=0%] (sum=142%)
38-
replicas#1: first: [s1=56, s2=44, s3=25, s4=16, s5=11, s6=9, s7=8, s8=8, s9=7, s10=8, s11=7, s12=7, s13=7, s14=7, s15=7, s16=7, s17=7, s18=7, s19=7, s20=8, s21=8, s22=7, s23=8, s24=7, s25=7] (stddev=11.96, mean=12.00, sum=300)
39-
replicas#1: last: [s1=55, s2=32, s3=15, s4=14, s5=11, s6=9, s7=9, s8=9, s9=9, s10=9, s11=8, s12=9, s13=8, s14=9, s15=8, s16=8, s17=8, s18=10, s19=9, s20=9, s21=8, s22=7, s23=9, s24=9, s25=9] (stddev=9.99, mean=12.00, sum=300)
40-
replicas#1: thrash_pct: [s1=0%, s2=0%, s3=0%, s4=0%, s5=4%, s6=0%, s7=0%, s8=0%, s9=0%, s10=0%, s11=0%, s12=0%, s13=0%, s14=0%, s15=4%, s16=0%, s17=0%, s18=0%, s19=0%, s20=0%, s21=0%, s22=0%, s23=0%, s24=0%, s25=0%] (sum=8%)
41-
write_bytes_per_second#1: last: [s1=9716, s2=9343, s3=9125, s4=9067, s5=9042, s6=9053, s7=9020, s8=9052, s9=9057, s10=9082, s11=9000, s12=9014, s13=9018, s14=9057, s15=9071, s16=9067, s17=9060, s18=9063, s19=9030, s20=9049, s21=9047, s22=9031, s23=9068, s24=9035, s25=9069] (stddev=142.27, mean=9089.44, sum=227236)
42-
write_bytes_per_second#1: thrash_pct: [s1=418%, s2=368%, s3=390%, s4=341%, s5=347%, s6=355%, s7=338%, s8=308%, s9=363%, s10=340%, s11=320%, s12=337%, s13=341%, s14=367%, s15=331%, s16=367%, s17=307%, s18=352%, s19=355%, s20=370%, s21=356%, s22=331%, s23=352%, s24=367%, s25=362%] (sum=8782%)
43-
artifacts[mma-only]: c30c22902539d7de
28+
cpu#1: last: [s1=463047636, s2=464742044, s3=465584574, s4=464309298, s5=465847383, s6=466444044, s7=463910580, s8=466175498, s9=464760769, s10=467967426, s11=468395602, s12=468693867, s13=468326290, s14=319992397, s15=468057733, s16=403289930, s17=467566175, s18=340531137, s19=467968120, s20=468090063, s21=361926413, s22=404937284, s23=469792185, s24=383090767, s25=319789535] (stddev=50329198.14, mean=437329470.00, sum=10933236750)
29+
cpu#1: thrash_pct: [s1=13%, s2=39%, s3=29%, s4=12%, s5=41%, s6=29%, s7=13%, s8=29%, s9=38%, s10=8%, s11=7%, s12=7%, s13=10%, s14=6%, s15=8%, s16=7%, s17=12%, s18=6%, s19=7%, s20=8%, s21=6%, s22=7%, s23=9%, s24=6%, s25=6%] (sum=362%)
30+
cpu_util#1: last: [s1=0.06, s2=0.06, s3=0.06, s4=0.06, s5=0.06, s6=0.06, s7=0.06, s8=0.06, s9=0.06, s10=0.06, s11=0.06, s12=0.06, s13=0.06, s14=0.04, s15=0.06, s16=0.05, s17=0.06, s18=0.04, s19=0.06, s20=0.06, s21=0.05, s22=0.05, s23=0.06, s24=0.05, s25=0.04] (stddev=0.01, mean=0.05, sum=1)
31+
cpu_util#1: thrash_pct: [s1=13%, s2=39%, s3=29%, s4=12%, s5=41%, s6=29%, s7=13%, s8=29%, s9=38%, s10=8%, s11=7%, s12=7%, s13=10%, s14=6%, s15=8%, s16=7%, s17=12%, s18=6%, s19=7%, s20=8%, s21=6%, s22=7%, s23=9%, s24=6%, s25=6%] (sum=362%)
32+
leases#1: first: [s1=167, s2=0, s3=0, s4=167, s5=0, s6=0, s7=166, s8=0, s9=0, s10=0, s11=0, s12=0, s13=0, s14=0, s15=0, s16=0, s17=0, s18=0, s19=0, s20=0, s21=0, s22=0, s23=0, s24=0, s25=0] (stddev=54.16, mean=20.00, sum=500)
33+
leases#1: last: [s1=21, s2=20, s3=20, s4=21, s5=20, s6=20, s7=21, s8=20, s9=20, s10=22, s11=22, s12=22, s13=22, s14=15, s15=22, s16=19, s17=22, s18=16, s19=22, s20=22, s21=17, s22=19, s23=22, s24=18, s25=15] (stddev=2.19, mean=20.00, sum=500)
34+
leases#1: thrash_pct: [s1=0%, s2=27%, s3=16%, s4=0%, s5=33%, s6=18%, s7=0%, s8=21%, s9=29%, s10=0%, s11=0%, s12=0%, s13=3%, s14=0%, s15=0%, s16=0%, s17=5%, s18=0%, s19=0%, s20=2%, s21=0%, s22=0%, s23=2%, s24=0%, s25=0%] (sum=156%)
35+
replicas#1: first: [s1=167, s2=167, s3=167, s4=167, s5=167, s6=167, s7=166, s8=166, s9=166, s10=0, s11=0, s12=0, s13=0, s14=0, s15=0, s16=0, s17=0, s18=0, s19=0, s20=0, s21=0, s22=0, s23=0, s24=0, s25=0] (stddev=80.00, mean=60.00, sum=1500)
36+
replicas#1: last: [s1=84, s2=153, s3=156, s4=86, s5=155, s6=157, s7=85, s8=155, s9=152, s10=22, s11=22, s12=22, s13=22, s14=15, s15=22, s16=19, s17=22, s18=16, s19=22, s20=22, s21=17, s22=19, s23=22, s24=18, s25=15] (stddev=57.14, mean=60.00, sum=1500)
37+
replicas#1: thrash_pct: [s1=0%, s2=11%, s3=5%, s4=0%, s5=13%, s6=8%, s7=0%, s8=9%, s9=10%, s10=0%, s11=0%, s12=0%, s13=3%, s14=0%, s15=0%, s16=0%, s17=4%, s18=0%, s19=0%, s20=2%, s21=0%, s22=0%, s23=2%, s24=0%, s25=0%] (sum=67%)
38+
write_bytes_per_second#1: last: [s1=125, s2=229, s3=233, s4=128, s5=231, s6=235, s7=126, s8=232, s9=227, s10=32, s11=33, s12=33, s13=32, s14=22, s15=32, s16=28, s17=32, s18=23, s19=32, s20=32, s21=25, s22=28, s23=33, s24=26, s25=22] (stddev=85.61, mean=89.24, sum=2231)
39+
write_bytes_per_second#1: thrash_pct: [s1=179%, s2=262%, s3=281%, s4=185%, s5=247%, s6=266%, s7=180%, s8=250%, s9=247%, s10=29%, s11=23%, s12=8%, s13=31%, s14=24%, s15=22%, s16=10%, s17=27%, s18=23%, s19=34%, s20=22%, s21=12%, s22=19%, s23=33%, s24=58%, s25=21%] (sum=2492%)
40+
artifacts[mma-only]: 99cd3e4d17a5a700
4441
==========================
45-
cpu#1: last: [s1=4195620275, s2=4196803898, s3=4201453881, s4=4207023762, s5=4205406711, s6=4208677351, s7=4214607012, s8=4200332000, s9=4197470437, s10=4195295352, s11=4213016495, s12=4204698413, s13=4207405981, s14=4200445833, s15=4195950476, s16=4206976540, s17=4201083041, s18=4195551669, s19=4194449158, s20=4199778225, s21=4200014668, s22=4201038712, s23=4199242495, s24=4197916137, s25=4199865623] (stddev=5412725.77, mean=4201604965.80, sum=105040124145)
46-
cpu#1: thrash_pct: [s1=4%, s2=4%, s3=18%, s4=3%, s5=2%, s6=10%, s7=2%, s8=11%, s9=11%, s10=10%, s11=2%, s12=3%, s13=10%, s14=3%, s15=3%, s16=11%, s17=11%, s18=11%, s19=11%, s20=11%, s21=2%, s22=11%, s23=2%, s24=2%, s25=3%] (sum=172%)
47-
cpu_util#1: last: [s1=0.52, s2=0.52, s3=0.53, s4=0.53, s5=0.53, s6=0.53, s7=0.53, s8=0.53, s9=0.52, s10=0.52, s11=0.53, s12=0.53, s13=0.53, s14=0.53, s15=0.52, s16=0.53, s17=0.53, s18=0.52, s19=0.52, s20=0.52, s21=0.53, s22=0.53, s23=0.52, s24=0.52, s25=0.52] (stddev=0.00, mean=0.53, sum=13)
48-
cpu_util#1: thrash_pct: [s1=4%, s2=4%, s3=18%, s4=3%, s5=2%, s6=10%, s7=2%, s8=11%, s9=11%, s10=10%, s11=2%, s12=3%, s13=10%, s14=3%, s15=3%, s16=11%, s17=11%, s18=11%, s19=11%, s20=11%, s21=2%, s22=11%, s23=2%, s24=2%, s25=3%] (sum=172%)
49-
leases#1: first: [s1=29, s2=13, s3=4, s4=2, s5=2, s6=2, s7=3, s8=3, s9=2, s10=3, s11=4, s12=1, s13=2, s14=2, s15=4, s16=3, s17=3, s18=3, s19=2, s20=3, s21=3, s22=2, s23=2, s24=1, s25=2] (stddev=5.56, mean=4.00, sum=100)
50-
leases#1: last: [s1=3, s2=3, s3=2, s4=3, s5=4, s6=4, s7=5, s8=4, s9=4, s10=4, s11=5, s12=5, s13=3, s14=5, s15=5, s16=4, s17=4, s18=6, s19=4, s20=4, s21=4, s22=4, s23=4, s24=3, s25=4] (stddev=0.85, mean=4.00, sum=100)
51-
leases#1: thrash_pct: [s1=0%, s2=9%, s3=15%, s4=0%, s5=0%, s6=8%, s7=0%, s8=7%, s9=8%, s10=7%, s11=0%, s12=0%, s13=7%, s14=0%, s15=0%, s16=7%, s17=14%, s18=8%, s19=8%, s20=7%, s21=0%, s22=8%, s23=0%, s24=0%, s25=0%] (sum=114%)
52-
replicas#1: first: [s1=56, s2=44, s3=25, s4=16, s5=11, s6=9, s7=8, s8=8, s9=7, s10=8, s11=7, s12=7, s13=7, s14=7, s15=7, s16=7, s17=7, s18=7, s19=7, s20=8, s21=8, s22=7, s23=8, s24=7, s25=7] (stddev=11.96, mean=12.00, sum=300)
53-
replicas#1: last: [s1=14, s2=12, s3=11, s4=12, s5=12, s6=12, s7=12, s8=12, s9=12, s10=12, s11=12, s12=13, s13=12, s14=12, s15=12, s16=12, s17=11, s18=11, s19=12, s20=12, s21=12, s22=11, s23=11, s24=12, s25=14] (stddev=0.75, mean=12.00, sum=300)
54-
replicas#1: thrash_pct: [s1=0%, s2=16%, s3=6%, s4=5%, s5=0%, s6=0%, s7=5%, s8=0%, s9=5%, s10=0%, s11=0%, s12=0%, s13=0%, s14=0%, s15=0%, s16=5%, s17=9%, s18=5%, s19=5%, s20=0%, s21=0%, s22=0%, s23=0%, s24=0%, s25=0%] (sum=60%)
55-
write_bytes_per_second#1: last: [s1=1687, s2=4653, s3=1641, s4=1668, s5=9057, s6=9096, s7=10547, s8=10574, s9=10578, s10=10615, s11=10528, s12=13516, s13=10549, s14=13531, s15=12095, s16=12095, s17=7615, s18=12030, s19=9073, s20=9093, s21=9109, s22=10572, s23=9099, s24=9085, s25=9144] (stddev=3277.31, mean=9090.00, sum=227250)
56-
write_bytes_per_second#1: thrash_pct: [s1=23%, s2=95%, s3=22%, s4=19%, s5=39%, s6=66%, s7=66%, s8=43%, s9=73%, s10=43%, s11=40%, s12=59%, s13=125%, s14=53%, s15=43%, s16=50%, s17=90%, s18=47%, s19=67%, s20=42%, s21=62%, s22=42%, s23=42%, s24=97%, s25=42%] (sum=1391%)
57-
artifacts[mma-count]: e82e4e9c52d587b
42+
cpu#1: last: [s1=479911909, s2=373580982, s3=417004825, s4=481191026, s5=437968812, s6=418529844, s7=480521667, s8=416565076, s9=375203500, s10=416487421, s11=459073056, s12=417390139, s13=458452447, s14=438198669, s15=438626986, s16=436488614, s17=459362930, s18=437947945, s19=458740016, s20=438620980, s21=437601149, s22=458747503, s23=416674300, s24=436424336, s25=460194196] (stddev=27439196.31, mean=437980333.12, sum=10949508328)
43+
cpu#1: thrash_pct: [s1=7%, s2=8%, s3=7%, s4=8%, s5=8%, s6=7%, s7=8%, s8=7%, s9=7%, s10=13%, s11=7%, s12=7%, s13=10%, s14=5%, s15=9%, s16=12%, s17=6%, s18=6%, s19=10%, s20=8%, s21=10%, s22=7%, s23=7%, s24=7%, s25=6%] (sum=199%)
44+
cpu_util#1: last: [s1=0.06, s2=0.05, s3=0.05, s4=0.06, s5=0.05, s6=0.05, s7=0.06, s8=0.05, s9=0.05, s10=0.05, s11=0.06, s12=0.05, s13=0.06, s14=0.05, s15=0.05, s16=0.05, s17=0.06, s18=0.05, s19=0.06, s20=0.05, s21=0.05, s22=0.06, s23=0.05, s24=0.05, s25=0.06] (stddev=0.00, mean=0.05, sum=1)
45+
cpu_util#1: thrash_pct: [s1=7%, s2=8%, s3=7%, s4=8%, s5=8%, s6=7%, s7=8%, s8=7%, s9=7%, s10=13%, s11=7%, s12=7%, s13=10%, s14=5%, s15=9%, s16=12%, s17=6%, s18=6%, s19=10%, s20=8%, s21=10%, s22=7%, s23=7%, s24=7%, s25=6%] (sum=199%)
46+
leases#1: first: [s1=167, s2=0, s3=0, s4=167, s5=0, s6=0, s7=166, s8=0, s9=0, s10=0, s11=0, s12=0, s13=0, s14=0, s15=0, s16=0, s17=0, s18=0, s19=0, s20=0, s21=0, s22=0, s23=0, s24=0, s25=0] (stddev=54.16, mean=20.00, sum=500)
47+
leases#1: last: [s1=22, s2=17, s3=19, s4=22, s5=20, s6=19, s7=22, s8=19, s9=17, s10=19, s11=21, s12=19, s13=21, s14=20, s15=20, s16=20, s17=21, s18=20, s19=21, s20=20, s21=20, s22=21, s23=19, s24=20, s25=21] (stddev=1.30, mean=20.00, sum=500)
48+
leases#1: thrash_pct: [s1=0%, s2=2%, s3=0%, s4=0%, s5=2%, s6=0%, s7=0%, s8=0%, s9=0%, s10=8%, s11=2%, s12=2%, s13=4%, s14=0%, s15=4%, s16=7%, s17=0%, s18=0%, s19=5%, s20=3%, s21=6%, s22=2%, s23=2%, s24=2%, s25=0%] (sum=50%)
49+
replicas#1: first: [s1=167, s2=167, s3=167, s4=167, s5=167, s6=167, s7=166, s8=166, s9=166, s10=0, s11=0, s12=0, s13=0, s14=0, s15=0, s16=0, s17=0, s18=0, s19=0, s20=0, s21=0, s22=0, s23=0, s24=0, s25=0] (stddev=80.00, mean=60.00, sum=1500)
50+
replicas#1: last: [s1=61, s2=60, s3=61, s4=63, s5=58, s6=63, s7=63, s8=60, s9=61, s10=58, s11=58, s12=59, s13=58, s14=61, s15=60, s16=57, s17=62, s18=61, s19=59, s20=60, s21=58, s22=60, s23=59, s24=58, s25=62] (stddev=1.74, mean=60.00, sum=1500)
51+
replicas#1: thrash_pct: [s1=0%, s2=0%, s3=0%, s4=0%, s5=0%, s6=0%, s7=0%, s8=0%, s9=0%, s10=0%, s11=0%, s12=0%, s13=0%, s14=0%, s15=5%, s16=0%, s17=0%, s18=0%, s19=0%, s20=0%, s21=0%, s22=0%, s23=0%, s24=0%, s25=2%] (sum=7%)
52+
write_bytes_per_second#1: last: [s1=91, s2=89, s3=91, s4=94, s5=87, s6=94, s7=94, s8=89, s9=91, s10=87, s11=86, s12=88, s13=87, s14=91, s15=89, s16=85, s17=92, s18=91, s19=88, s20=90, s21=87, s22=90, s23=88, s24=87, s25=92] (stddev=2.50, mean=89.52, sum=2238)
53+
write_bytes_per_second#1: thrash_pct: [s1=74%, s2=98%, s3=89%, s4=68%, s5=111%, s6=106%, s7=63%, s8=104%, s9=110%, s10=45%, s11=38%, s12=48%, s13=34%, s14=33%, s15=81%, s16=44%, s17=47%, s18=43%, s19=32%, s20=68%, s21=37%, s22=67%, s23=40%, s24=70%, s25=55%] (sum=1606%)
54+
artifacts[mma-count]: 783f63043b18076a
5855
==========================

0 commit comments

Comments
 (0)