Increase load benchmark repetitions to 5 #9898

sarahchen6 · 2025-11-03T20:34:38Z

work in progress!

What Does This Do

Separate the petclinic and insecure-bank load benchmarks into two separate jobs that run in parallel in order to reduce CI time while increasing the repetition count to 5.

Motivation

Improve load benchmark stability without increasing CI time.

Additional Notes

It looks like the benchmarks do stabilize with increased repetitions: https://benchmarking.us1.prod.dog/benchmarks?ciJobDateStart=1761070654154&ciJobDateEnd=1763752654154&projectId=4&projectId=46&projectId=45&gitBranch=sarahchen6%2Fincrease-load-benchmark-reps&page=1

I'm holding off on requesting a review for this though because it would severely increase CI time.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: https://datadoghq.atlassian.net/browse/LANGPLAT-624

pr-commenter · 2025-11-04T23:17:55Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/increase-load-benchmark-reps
git_commit_date	1763731854	1763739553
git_commit_sha	`614cc72`	`f3f7d5c`
release_version	1.57.0-SNAPSHOT~614cc721f6	1.56.0-SNAPSHOT~f3f7d5cbad

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1763741461	1763741461
ci_job_id	1251033274	1251033274
ci_pipeline_id	83611900	83611900
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-gfqb0ewp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-gfqb0ewp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 5 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~f3f7d5cbad, baseline=1.57.0-SNAPSHOT~614cc721f6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.099 s) : 0, 1099101
Total [baseline] (8.857 s) : 0, 8856520
Agent [candidate] (1.109 s) : 0, 1109258
Total [candidate] (8.895 s) : 0, 8895034
section iast
Agent [baseline] (1.242 s) : 0, 1242157
Total [baseline] (9.563 s) : 0, 9563025
Agent [candidate] (1.242 s) : 0, 1242024
Total [candidate] (9.569 s) : 0, 9569163

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.099 s	-
Agent	iast	1.242 s	143.056 ms (13.0%)
Total	tracing	8.857 s	-
Total	iast	9.563 s	706.505 ms (8.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.109 s	-
Agent	iast	1.242 s	132.766 ms (12.0%)
Total	tracing	8.895 s	-
Total	iast	9.569 s	674.129 ms (7.6%)

gantt
    title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~f3f7d5cbad, baseline=1.57.0-SNAPSHOT~614cc721f6

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.499 ms) : 0, 1499
crashtracking [candidate] (1.493 ms) : 0, 1493
BytebuddyAgent [baseline] (706.643 ms) : 0, 706643
BytebuddyAgent [candidate] (713.515 ms) : 0, 713515
GlobalTracer [baseline] (248.319 ms) : 0, 248319
GlobalTracer [candidate] (250.788 ms) : 0, 250788
AppSec [baseline] (31.848 ms) : 0, 31848
AppSec [candidate] (32.183 ms) : 0, 32183
Debugger [baseline] (63.292 ms) : 0, 63292
Debugger [candidate] (63.557 ms) : 0, 63557
Remote Config [baseline] (638.088 µs) : 0, 638
Remote Config [candidate] (643.938 µs) : 0, 644
Telemetry [baseline] (8.191 ms) : 0, 8191
Telemetry [candidate] (8.255 ms) : 0, 8255
Flare Poller [baseline] (3.644 ms) : 0, 3644
Flare Poller [candidate] (3.747 ms) : 0, 3747
section iast
crashtracking [baseline] (1.485 ms) : 0, 1485
crashtracking [candidate] (1.492 ms) : 0, 1492
BytebuddyAgent [baseline] (833.297 ms) : 0, 833297
BytebuddyAgent [candidate] (835.991 ms) : 0, 835991
GlobalTracer [baseline] (238.066 ms) : 0, 238066
GlobalTracer [candidate] (237.22 ms) : 0, 237220
IAST [baseline] (28.38 ms) : 0, 28380
IAST [candidate] (28.837 ms) : 0, 28837
AppSec [baseline] (33.205 ms) : 0, 33205
AppSec [candidate] (31.919 ms) : 0, 31919
Debugger [baseline] (61.031 ms) : 0, 61031
Debugger [candidate] (59.944 ms) : 0, 59944
Remote Config [baseline] (561.826 µs) : 0, 562
Remote Config [candidate] (539.381 µs) : 0, 539
Telemetry [baseline] (7.715 ms) : 0, 7715
Telemetry [candidate] (7.568 ms) : 0, 7568
Flare Poller [baseline] (3.471 ms) : 0, 3471
Flare Poller [candidate] (3.507 ms) : 0, 3507

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~f3f7d5cbad, baseline=1.57.0-SNAPSHOT~614cc721f6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.106 s) : 0, 1105989
Total [baseline] (10.871 s) : 0, 10871132
Agent [candidate] (1.109 s) : 0, 1109124
Total [candidate] (10.822 s) : 0, 10821608
section appsec
Agent [baseline] (1.287 s) : 0, 1287079
Total [baseline] (11.071 s) : 0, 11070568
Agent [candidate] (1.285 s) : 0, 1285078
Total [candidate] (11.148 s) : 0, 11148445
section iast
Agent [baseline] (1.245 s) : 0, 1244828
Total [baseline] (11.319 s) : 0, 11319270
Agent [candidate] (1.257 s) : 0, 1256689
Total [candidate] (11.354 s) : 0, 11354170
section profiling
Agent [baseline] (1.243 s) : 0, 1242830
Total [baseline] (11.14 s) : 0, 11139671
Agent [candidate] (1.243 s) : 0, 1243029
Total [candidate] (11.157 s) : 0, 11157246

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.106 s	-
Agent	appsec	1.287 s	181.09 ms (16.4%)
Agent	iast	1.245 s	138.839 ms (12.6%)
Agent	profiling	1.243 s	136.841 ms (12.4%)
Total	tracing	10.871 s	-
Total	appsec	11.071 s	199.435 ms (1.8%)
Total	iast	11.319 s	448.137 ms (4.1%)
Total	profiling	11.14 s	268.538 ms (2.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.109 s	-
Agent	appsec	1.285 s	175.954 ms (15.9%)
Agent	iast	1.257 s	147.565 ms (13.3%)
Agent	profiling	1.243 s	133.905 ms (12.1%)
Total	tracing	10.822 s	-
Total	appsec	11.148 s	326.837 ms (3.0%)
Total	iast	11.354 s	532.562 ms (4.9%)
Total	profiling	11.157 s	335.638 ms (3.1%)

gantt
    title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~f3f7d5cbad, baseline=1.57.0-SNAPSHOT~614cc721f6

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.496 ms) : 0, 1496
crashtracking [candidate] (1.502 ms) : 0, 1502
BytebuddyAgent [baseline] (710.137 ms) : 0, 710137
BytebuddyAgent [candidate] (713.836 ms) : 0, 713836
GlobalTracer [baseline] (249.768 ms) : 0, 249768
GlobalTracer [candidate] (249.703 ms) : 0, 249703
AppSec [baseline] (32.17 ms) : 0, 32170
AppSec [candidate] (32.053 ms) : 0, 32053
Debugger [baseline] (64.681 ms) : 0, 64681
Debugger [candidate] (64.368 ms) : 0, 64368
Remote Config [baseline] (641.8 µs) : 0, 642
Remote Config [candidate] (629.218 µs) : 0, 629
Telemetry [baseline] (8.457 ms) : 0, 8457
Telemetry [candidate] (8.318 ms) : 0, 8318
Flare Poller [baseline] (3.802 ms) : 0, 3802
Flare Poller [candidate] (3.753 ms) : 0, 3753
section appsec
crashtracking [baseline] (1.492 ms) : 0, 1492
crashtracking [candidate] (1.476 ms) : 0, 1476
BytebuddyAgent [baseline] (734.246 ms) : 0, 734246
BytebuddyAgent [candidate] (733.878 ms) : 0, 733878
GlobalTracer [baseline] (241.825 ms) : 0, 241825
GlobalTracer [candidate] (241.354 ms) : 0, 241354
IAST [baseline] (24.906 ms) : 0, 24906
IAST [candidate] (24.973 ms) : 0, 24973
AppSec [baseline] (175.218 ms) : 0, 175218
AppSec [candidate] (174.039 ms) : 0, 174039
Debugger [baseline] (61.384 ms) : 0, 61384
Debugger [candidate] (61.564 ms) : 0, 61564
Remote Config [baseline] (674.106 µs) : 0, 674
Remote Config [candidate] (662.53 µs) : 0, 663
Telemetry [baseline] (8.305 ms) : 0, 8305
Telemetry [candidate] (8.106 ms) : 0, 8106
Flare Poller [baseline] (4.006 ms) : 0, 4006
Flare Poller [candidate] (4.016 ms) : 0, 4016
section iast
crashtracking [baseline] (1.49 ms) : 0, 1490
crashtracking [candidate] (1.501 ms) : 0, 1501
BytebuddyAgent [baseline] (835.297 ms) : 0, 835297
BytebuddyAgent [candidate] (844.387 ms) : 0, 844387
GlobalTracer [baseline] (238.398 ms) : 0, 238398
GlobalTracer [candidate] (239.978 ms) : 0, 239978
IAST [baseline] (28.617 ms) : 0, 28617
IAST [candidate] (28.077 ms) : 0, 28077
AppSec [baseline] (33.215 ms) : 0, 33215
AppSec [candidate] (34.285 ms) : 0, 34285
Debugger [baseline] (61.164 ms) : 0, 61164
Debugger [candidate] (61.479 ms) : 0, 61479
Remote Config [baseline] (543.928 µs) : 0, 544
Remote Config [candidate] (550.526 µs) : 0, 551
Telemetry [baseline] (7.632 ms) : 0, 7632
Telemetry [candidate] (7.655 ms) : 0, 7655
Flare Poller [baseline] (3.524 ms) : 0, 3524
Flare Poller [candidate] (3.538 ms) : 0, 3538
section profiling
crashtracking [baseline] (1.444 ms) : 0, 1444
crashtracking [candidate] (1.436 ms) : 0, 1436
BytebuddyAgent [baseline] (740.439 ms) : 0, 740439
BytebuddyAgent [candidate] (741.851 ms) : 0, 741851
GlobalTracer [baseline] (224.553 ms) : 0, 224553
GlobalTracer [candidate] (224.234 ms) : 0, 224234
AppSec [baseline] (32.641 ms) : 0, 32641
AppSec [candidate] (32.534 ms) : 0, 32534
Debugger [baseline] (63.907 ms) : 0, 63907
Debugger [candidate] (63.401 ms) : 0, 63401
Remote Config [baseline] (665.64 µs) : 0, 666
Remote Config [candidate] (647.15 µs) : 0, 647
Telemetry [baseline] (8.055 ms) : 0, 8055
Telemetry [candidate] (8.014 ms) : 0, 8014
Flare Poller [baseline] (3.797 ms) : 0, 3797
Flare Poller [candidate] (3.736 ms) : 0, 3736
ProfilingAgent [baseline] (97.866 ms) : 0, 97866
ProfilingAgent [candidate] (97.642 ms) : 0, 97642
Profiling [baseline] (98.45 ms) : 0, 98450
Profiling [candidate] (98.219 ms) : 0, 98219

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/increase-load-benchmark-reps
git_commit_date	1763731854	1763739553
git_commit_sha	`614cc72`	`f3f7d5c`
release_version	1.57.0-SNAPSHOT~614cc721f6	1.56.0-SNAPSHOT~f3f7d5cbad

See matching parameters

	Baseline	Candidate
application	petclinic	petclinic
ci_job_date	1763742305	1763742305
ci_job_id	1251033277	1251033277
ci_pipeline_id	83611900	83611900
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-t5b1n1hb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-t5b1n1hb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 6 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~f3f7d5cbad, baseline=1.57.0-SNAPSHOT~614cc721f6
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.768 ms) : 18646, 18890
.   : milestone, 18768,
appsec (18.947 ms) : 18825, 19068
.   : milestone, 18947,
code_origins (17.771 ms) : 17658, 17884
.   : milestone, 17771,
iast (17.669 ms) : 17557, 17781
.   : milestone, 17669,
profiling (18.655 ms) : 18539, 18772
.   : milestone, 18655,
tracing (17.54 ms) : 17431, 17649
.   : milestone, 17540,
section candidate
no_agent (18.437 ms) : 18317, 18557
.   : milestone, 18437,
appsec (18.948 ms) : 18826, 19070
.   : milestone, 18948,
code_origins (18.025 ms) : 17911, 18140
.   : milestone, 18025,
iast (17.838 ms) : 17724, 17951
.   : milestone, 17838,
profiling (19.028 ms) : 18906, 19149
.   : milestone, 19028,
tracing (17.505 ms) : 17396, 17614
.   : milestone, 17505,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.768 ms [18.646 ms, 18.89 ms]	-
appsec	18.947 ms [18.825 ms, 19.068 ms]	178.85 µs (1.0%)
code_origins	17.771 ms [17.658 ms, 17.884 ms]	-996.94 µs (-5.3%)
iast	17.669 ms [17.557 ms, 17.781 ms]	-1.099 ms (-5.9%)
profiling	18.655 ms [18.539 ms, 18.772 ms]	-112.838 µs (-0.6%)
tracing	17.54 ms [17.431 ms, 17.649 ms]	-1.228 ms (-6.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.437 ms [18.317 ms, 18.557 ms]	-
appsec	18.948 ms [18.826 ms, 19.07 ms]	511.175 µs (2.8%)
code_origins	18.025 ms [17.911 ms, 18.14 ms]	-411.804 µs (-2.2%)
iast	17.838 ms [17.724 ms, 17.951 ms]	-599.429 µs (-3.3%)
profiling	19.028 ms [18.906 ms, 19.149 ms]	590.414 µs (3.2%)
tracing	17.505 ms [17.396 ms, 17.614 ms]	-932.089 µs (-5.1%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/increase-load-benchmark-reps
git_commit_date	1763731854	1763739553
git_commit_sha	`614cc72`	`f3f7d5c`
release_version	1.57.0-SNAPSHOT~614cc721f6	1.56.0-SNAPSHOT~f3f7d5cbad

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1763741736	1763741736
ci_job_id	1251033293	1251033293
ci_pipeline_id	83611900	83611900
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-6-zlk98uvn 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-6-zlk98uvn 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~f3f7d5cbad, baseline=1.57.0-SNAPSHOT~614cc721f6
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.557 s) : 15557000, 15557000
.   : milestone, 15557000,
appsec (15.023 s) : 15023000, 15023000
.   : milestone, 15023000,
iast (18.035 s) : 18035000, 18035000
.   : milestone, 18035000,
iast_GLOBAL (18.274 s) : 18274000, 18274000
.   : milestone, 18274000,
profiling (14.574 s) : 14574000, 14574000
.   : milestone, 14574000,
tracing (14.656 s) : 14656000, 14656000
.   : milestone, 14656000,
section candidate
no_agent (15.497 s) : 15497000, 15497000
.   : milestone, 15497000,
appsec (14.598 s) : 14598000, 14598000
.   : milestone, 14598000,
iast (18.724 s) : 18724000, 18724000
.   : milestone, 18724000,
iast_GLOBAL (17.888 s) : 17888000, 17888000
.   : milestone, 17888000,
profiling (14.713 s) : 14713000, 14713000
.   : milestone, 14713000,
tracing (15.044 s) : 15044000, 15044000
.   : milestone, 15044000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.557 s [15.557 s, 15.557 s]	-
appsec	15.023 s [15.023 s, 15.023 s]	-534.0 ms (-3.4%)
iast	18.035 s [18.035 s, 18.035 s]	2.478 s (15.9%)
iast_GLOBAL	18.274 s [18.274 s, 18.274 s]	2.717 s (17.5%)
profiling	14.574 s [14.574 s, 14.574 s]	-983.0 ms (-6.3%)
tracing	14.656 s [14.656 s, 14.656 s]	-901.0 ms (-5.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.497 s [15.497 s, 15.497 s]	-
appsec	14.598 s [14.598 s, 14.598 s]	-899.0 ms (-5.8%)
iast	18.724 s [18.724 s, 18.724 s]	3.227 s (20.8%)
iast_GLOBAL	17.888 s [17.888 s, 17.888 s]	2.391 s (15.4%)
profiling	14.713 s [14.713 s, 14.713 s]	-784.0 ms (-5.1%)
tracing	15.044 s [15.044 s, 15.044 s]	-453.0 ms (-2.9%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~f3f7d5cbad, baseline=1.57.0-SNAPSHOT~614cc721f6
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.481 ms) : 1469, 1493
.   : milestone, 1481,
appsec (3.657 ms) : 3442, 3873
.   : milestone, 3657,
iast (2.231 ms) : 2166, 2296
.   : milestone, 2231,
iast_GLOBAL (2.27 ms) : 2205, 2335
.   : milestone, 2270,
profiling (2.109 ms) : 2055, 2164
.   : milestone, 2109,
tracing (2.06 ms) : 2009, 2111
.   : milestone, 2060,
section candidate
no_agent (1.488 ms) : 1477, 1500
.   : milestone, 1488,
appsec (3.733 ms) : 3512, 3954
.   : milestone, 3733,
iast (2.225 ms) : 2161, 2290
.   : milestone, 2225,
iast_GLOBAL (2.272 ms) : 2207, 2337
.   : milestone, 2272,
profiling (2.49 ms) : 2330, 2651
.   : milestone, 2490,
tracing (2.052 ms) : 2001, 2103
.   : milestone, 2052,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.481 ms [1.469 ms, 1.493 ms]	-
appsec	3.657 ms [3.442 ms, 3.873 ms]	2.176 ms (147.0%)
iast	2.231 ms [2.166 ms, 2.296 ms]	750.008 µs (50.6%)
iast_GLOBAL	2.27 ms [2.205 ms, 2.335 ms]	789.017 µs (53.3%)
profiling	2.109 ms [2.055 ms, 2.164 ms]	628.407 µs (42.4%)
tracing	2.06 ms [2.009 ms, 2.111 ms]	578.761 µs (39.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.488 ms [1.477 ms, 1.5 ms]	-
appsec	3.733 ms [3.512 ms, 3.954 ms]	2.244 ms (150.8%)
iast	2.225 ms [2.161 ms, 2.29 ms]	737.032 µs (49.5%)
iast_GLOBAL	2.272 ms [2.207 ms, 2.337 ms]	783.475 µs (52.6%)
profiling	2.49 ms [2.33 ms, 2.651 ms]	1.002 ms (67.3%)
tracing	2.052 ms [2.001 ms, 2.103 ms]	563.463 µs (37.9%)

This reverts commit 030816f.

sarahchen6 requested a review from a team as a code owner November 3, 2025 20:34

sarahchen6 requested review from amarziali and removed request for a team November 3, 2025 20:34

sarahchen6 added type: enhancement Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Nov 3, 2025

This comment has been minimized.

Sign in to view

sarahchen6 marked this pull request as draft November 3, 2025 22:08

sarahchen6 force-pushed the sarahchen6/increase-load-benchmark-reps branch from f7ce7fb to f68d358 Compare November 4, 2025 21:32

sarahchen6 force-pushed the sarahchen6/increase-load-benchmark-reps branch from f68d358 to 140dd23 Compare November 6, 2025 17:51

sarahchen6 added 7 commits November 10, 2025 16:52

Increase load benchmark repetition count to 5

ab99f6d

Increase timeout to 2h

fb44c5d

Attempt to parallelize load benchmarks

59bfbda

Parallelize into separate jobs

a9a53c6

Add missing load benchmark split

bbd98ce

Empty commit to run CI benchmarks again

0bdbb63

Run load benchmarks 5 times in parallel

9e108de

sarahchen6 force-pushed the sarahchen6/increase-load-benchmark-reps branch from eec9091 to 9e108de Compare November 10, 2025 21:53

sarahchen6 added 9 commits November 11, 2025 12:13

Merge branch 'master' into sarahchen6/increase-load-benchmark-reps

4243ccc

Merge branch 'master' into sarahchen6/increase-load-benchmark-reps

f17a248

Increase repetitions to 7

b69f027

Change repetitions back to 5

6bd024e

Merge branch 'master' into sarahchen6/increase-load-benchmark-reps

255b89c

Merge branch 'master' into sarahchen6/increase-load-benchmark-reps

c486f67

Analyze results at end

030816f

Revert "Analyze results at end"

06ba98e

This reverts commit 030816f.

Merge branch 'master' into sarahchen6/increase-load-benchmark-reps

f3f7d5c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Increase load benchmark repetitions to 5 #9898

Increase load benchmark repetitions to 5 #9898

sarahchen6 commented Nov 3, 2025 •

edited

Loading

Uh oh!

This comment has been minimized.

pr-commenter bot commented Nov 4, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Increase load benchmark repetitions to 5 #9898

Are you sure you want to change the base?

Increase load benchmark repetitions to 5 #9898

Conversation

sarahchen6 commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

This comment has been minimized.

pr-commenter bot commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sarahchen6 commented Nov 3, 2025 •

edited

Loading

pr-commenter bot commented Nov 4, 2025 •

edited

Loading