Skip to content

Commit 19223af

Browse files
committed
asim: fix shorter than expected replica change delay
Previously, we used integer division to compute the delay for adding a replica, which led to shorter than expected delays. We previously attempted to fix this by switching to float division but didn't notice we are later casting to time.Duration which truncated the result. This commit fixes the issue by casting time.Second to a float instead. Before: ``` I250709 03:24:05.016545 47 kv/kvserver/asim/op/settings.go:158 ⋮ [-] 54117 estimatedTimeToAddReplica: 0.80, s.RebalancingSnapshotRate: 33554432, base: 100000000 I250709 03:24:05.016549 47 kv/kvserver/asim/op/settings.go:161 ⋮ [-] 54118 calculated replica change delay: ‹100ms› for range size 26843545 ``` After: ``` I250709 03:27:53.914048 43 kv/kvserver/asim/op/settings.go:159 ⋮ [-] 11096 estimatedTimeToAddReplica: 0.80, s.RebalancingSnapshotRate: 33554432, base: 100000000 I250709 03:27:53.914053 43 kv/kvserver/asim/op/settings.go:162 ⋮ [-] 11097 calculated replica change delay: ‹899.999982ms› for range size 26843545 ``` See slack https://cockroachlabs.slack.com/archives/C048HDZJSAY/p1751935031605719 for more context. Epic: none Release note: none
1 parent 658969b commit 19223af

File tree

3 files changed

+44
-42
lines changed

3 files changed

+44
-42
lines changed

pkg/kv/kvserver/asim/config/settings.go

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,9 @@ func (s *SimulationSettings) ReplicaChangeDelayFn() func(rangeSize int64, add bo
163163
delay := s.ReplicaChangeBaseDelay
164164
if add {
165165
estimatedTimeToAddReplica := float64(rangeSize) / float64(s.RebalancingSnapshotRate)
166-
delay += time.Duration(estimatedTimeToAddReplica) * time.Second
166+
// To avoid truncating the decimal part of the result, cast to Duration
167+
// after float multiplication.
168+
delay += time.Duration(estimatedTimeToAddReplica * float64(time.Second))
167169
}
168170
return delay
169171
}

pkg/kv/kvserver/asim/tests/testdata/non_rand/example_add_node.txt

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -37,18 +37,18 @@ plot stat=replicas sample=1
3737
----
3838
301 ┼──────────────────────────────────────╭────────────────────────────────────────
3939
281 ┤ ╭──╯
40-
261 ┤ ╭──╯╯
41-
241 ┤ ╭──
42-
221 ┤ ╭──╯─╯
43-
201 ┤ ╭─╯─╯
44-
181 ┤ ╭─╯─╯
45-
161 ┤ ╭──╯
46-
140 ┤ ╭─
47-
120 ┤ ╭─╯
48-
100 ┤ ╭──╯
49-
80 ┤ ╭─
50-
60 ┤ ╭─╯
51-
40 ┤ ╭─
40+
261 ┤ ─╯╯
41+
241 ┤ ╭──╯
42+
221 ┤ ╭╭─╯
43+
201 ┤ ╭╭──╯
44+
181 ┤ ╭─╯
45+
161 ┤ ╭╭──
46+
140 ┤ ╭─
47+
120 ┤ ╭─╯
48+
100 ┤ ╭──╯
49+
80 ┤ ╭─
50+
60 ┤ ╭─╯
51+
40 ┤ ╭─
5252
20 ┤ ╭──╯
5353
0 ┼─╯
5454
replicas

pkg/kv/kvserver/asim/tests/testdata/non_rand/example_fulldisk.txt

Lines changed: 29 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -19,46 +19,46 @@ OK
1919
# store 5. This is shown below as it sheds replicas.
2020
plot stat=replicas
2121
----
22-
364╭╭╮╭─╮╮╭╭╮─╭───╮╭─╮╮╭─╭╮───╭╮─╭╭─╮─╭────╮╭╭───────╮──╭─╮───────────────
23-
344 ┤ ╭───╭─────╯╰─╯─────────╯╰───╯╰──╯╯╰─╯╰╰──╰─╯╯╰────╯╰──╯╯╰─╯╯ ╰───╯ ╰────
24-
324 ┤ ╭─╯╭───╯
25-
304 ┼─╮───╯
26-
283╰╮
27-
263
28-
243 ┤ ╰╮
29-
223 ┤ ╰╮
30-
203 ┤ ╰╮
31-
183 ┤ ╰╮
22+
368 ╭╮╭╮╮╭╮ ╭╮ ╭─╮╭───╮ ╭╭─╭─╮────╮──╭─╮────────╮──╭──╭──╮──
23+
348 ╭─────╭──────────────────────────────╯─╰────╯╰───╯─╰────╯╯──╰╰─────╯─╯╰──
24+
327 ┤ ╭╭─╯╯╭───╯ ╰╯ ╰╯ ╰─╯
25+
307 ┼─╮─────╯
26+
286
27+
266╰╮
28+
245 ┤ ╰╮
29+
225 ┤ ╰╮
30+
204 ┤ ╰╮
31+
184 ┤ ╰╮
3232
163 ┤ ╰╮
3333
143 ┤ │
34-
122 ┤ ╰╮
35-
102 ┤ ╰────────╮╭──╮╭─╮╭──╮ ╭──╮ ╭──╮ ╭
36-
82 ┤ ╰╯ ╰╯ ╰╯╯ ╰─╯ ╰─╯╰──────────────╮╭───╮╭──╮╭────╮ ╭──╮
37-
62 ╰╯ ╰╯ ╰╯ ╰─╯ ╰─
34+
122 ┤ ╰╮ ╭╮ ╭╮
35+
102 ┤ ╰──────╯╰╮│╰────╮╭──╮╭╮ ╭╮ ╭╮ ╭╮ ╭─
36+
81 ┤ ╰╯ ╰╯ ╰╯╰──╯╰──╯╰───────╯╰──╮│ ╰──╮╭───╮╭──╮╭──╮ ╭──╮ ╭─
37+
61╰╯ ╰╯ ╰╯ ╰╯ ╰─╯ ╰──╯
3838
replicas
3939
initial store values: [s1=300, s2=300, s3=300, s4=300, s5=300] (stddev=0.00, mean=300.00, sum=1500)
40-
last store values: [s1=363, s2=352, s3=360, s4=362, s5=66] (stddev=117.36, mean=300.60, sum=1503)
40+
last store values: [s1=354, s2=359, s3=343, s4=356, s5=88] (stddev=106.14, mean=300.00, sum=1500)
4141

4242
# Plot the % of disk storage capacity used. We should see s5 hovering right
4343
# around 92.5-95% (the storage capacity threshold value).
4444
plot stat=disk_fraction_used
4545
----
46-
2.51 ┼─╮
47-
2.37 ┤ ╰╮
48-
2.23 ┤ │
49-
2.09 ┤ ╰╮
50-
1.95 ┤ ╰╮
51-
1.81 ┤
46+
2.52 ┼─╮
47+
2.38 ┤ ╰╮
48+
2.24 ┤ │
49+
2.10 ┤ ╰╮
50+
1.95 ┤
51+
1.81 ┤ ╰╮
5252
1.67 ┤ ╰╮
5353
1.53 ┤ ╰╮
54-
1.39 ┤ ╰╮
55-
1.25 ┤ │ ╭╮ ╭╮ ╭─╮
56-
1.11 ┤ ╰╮ ╭╮ ╭╮ ╭──╮ │╰╮ ╭╮ ╭╮ ╭─╭──╮ ╭╮ │╰─╮ │ ╰╮ ╭─╮
57-
0.97 ┤ ╰╮╭───╮╭─╮╭──╮│╰╮╯╰─╮│ ╰╮│ ╰─╮│╰─╯╰╮│ ╰─╯ ╰─╮╭╯╰─╮│ ╰╮╭╯ ╰─╮│ ╰──
58-
0.83 ┤ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯╭─╭╰╯──╰╯─────╰╯────
59-
0.69 ┤ ╭╭─╭─╮╭───────────────────────╯╯╰─╯ ╰
60-
0.55 ┤ ╭──╭────────────────────────╯ ╰╯
54+
1.39 ┤ ╰╮ ╭╮
55+
1.25 ┤ │ ╭╮ ╭╮ │╰
56+
1.11 ┤ ╰╮ ╭╮ ╭╮ ╭╮ ╭╮ ╭╭╮ ╭╮ ╭──╮ │╰─╮ ╭──╮ ╭─╮ ╭╮ │╰─╮ │
57+
0.97 ┤ ╰╮╭───╮│╰╮│╰─────╯╰─╮│╰╮╭╯╰─╮│╰─╮╭╯╰╮│ ╰─╮│ ╰─╮│ ╰╮│ ╰╮╭╯╰─╮│ ╰╮╭╯
58+
0.83 ┤ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯ ╰╯╭─╭╰╯──╰╯───╰╯───╰╯──
59+
0.69 ┤ ╭╭─╮╭─────────────────────────╯╯╰─╯
60+
0.55 ┤ ╭─╭──────────────────────╯╯╰╯
6161
0.41 ┼────────╯
6262
disk_fraction_used
6363
initial store values: [s1=0.41, s2=0.41, s3=0.41, s4=0.41, s5=2.50] (stddev=0.84, mean=0.83, sum=4)
64-
last store values: [s1=0.87, s2=0.85, s3=0.87, s4=0.87, s5=0.97] (stddev=0.04, mean=0.89, sum=4)
64+
last store values: [s1=0.85, s2=0.87, s3=0.83, s4=0.86, s5=1.30] (stddev=0.18, mean=0.94, sum=5)

0 commit comments

Comments
 (0)