Nikita tkachenko/junit 6 #9264

nikita-tkachenko-datadog · 2025-07-28T18:09:55Z

What Does This Do

Migrates AgentTestRunner (base class for instrumentation tests) from JUnit4-based SpockRunner to JUnit5-based SpockExtension.

Motivation

Required for JDK 25 and JUnit 6 support.

The tracer uses a Spock version that runs on top of JUnit 5. We were using JUnitPlatformRunner to execute tests "in a JUnit 4 environment".
Starting from JUnit 6 the platform runner is no longer supported so we have to fully migrate to JUnit 5.

Additional Notes

Instrumentation tests need to patch the bootstrap classpath adding some core tracer classes to it.
The classes added to the bootstrap classpath cannot be loaded before the classpath patching takes place - if they're loaded by the application classloader first, then these application CP versions will shadow the bootstrap ones.

When running instrumentation tests with JUnit 5 this becomes a problem: the framework scans classpath to determine which tests to run. When scanning the classpath, it loads classes - including the classes should be appended to the bootstrap classpath.

To make sure classpath patching happens before scanning a custom org.junit.platform.launcher.LauncherSessionListener implementation was added: datadog.trace.agent.test.BootstrapClasspathSetup.
LauncherSessionListener implementations are discovered using Java ServiceLoader from META-INF/services/org.junit.platform.launcher.LauncherSessionListener files available on the classpath.
The overridden org.junit.platform.launcher.LauncherSessionListener#launcherSessionOpened method is executed early in the testing framework lifecycle - before the classpath scanning occurs.

The problem with this approach is that it will trigger classpath patching for every execution that has BootstrapClasspathSetup in the classpath.
This includes regular unit tests, where patching the bootstrap classpath caused failures in some cases.

To avoid this, BootstrapClasspathSetup was moved to a new Gradle module instrumentation-testing, which is only added to the dependencies of the instrumentation modules that need it.

There was another problem with the agent-iast module - it included testFixtures that depended on the AgentTestRunner, yet loading it (and triggering the classpath patch) caused the unit tests in this module to fail.
To mitigate this, testFixtures were moved to a separate new module - iast-testing.

The same had to be done with test fixtures in agent-civisbility and appsec modules - they were split into a separate civisibility-testing and appsec-testing modules.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

AlexeyKuznetsov-DD · 2025-07-28T18:23:35Z

You also may consider to delete spock-junit4 from libs.versions.toml catalog.

pr-commenter · 2025-07-28T18:42:43Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/junit-6
git_commit_date	1754661579	1754670324
git_commit_sha	`5ed2f59`	`7c7286a`
release_version	1.53.0-SNAPSHOT~5ed2f5961b	1.53.0-SNAPSHOT~7c7286a04f

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1754672101	1754672101
ci_job_id	1071882532	1071882532
ci_pipeline_id	73135578	73135578
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-60rfm48u 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-60rfm48u 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 13 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~7c7286a04f, baseline=1.53.0-SNAPSHOT~5ed2f5961b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.045 s) : 0, 1044507
Total [baseline] (10.72 s) : 0, 10719596
Agent [candidate] (1.048 s) : 0, 1047712
Total [candidate] (10.669 s) : 0, 10668689
section appsec
Agent [baseline] (1.217 s) : 0, 1217480
Total [baseline] (10.721 s) : 0, 10720973
Agent [candidate] (1.219 s) : 0, 1218515
Total [candidate] (10.742 s) : 0, 10741851
section iast
Agent [baseline] (1.183 s) : 0, 1182608
Total [baseline] (10.875 s) : 0, 10875136
Agent [candidate] (1.18 s) : 0, 1180411
Total [candidate] (10.936 s) : 0, 10935743
section profiling
Agent [baseline] (1.191 s) : 0, 1191377
Total [baseline] (10.814 s) : 0, 10813951
Agent [candidate] (1.202 s) : 0, 1201576
Total [candidate] (10.839 s) : 0, 10838941

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.045 s	-
Agent	appsec	1.217 s	172.973 ms (16.6%)
Agent	iast	1.183 s	138.101 ms (13.2%)
Agent	profiling	1.191 s	146.87 ms (14.1%)
Total	tracing	10.72 s	-
Total	appsec	10.721 s	1.378 ms (0.0%)
Total	iast	10.875 s	155.54 ms (1.5%)
Total	profiling	10.814 s	94.355 ms (0.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.048 s	-
Agent	appsec	1.219 s	170.803 ms (16.3%)
Agent	iast	1.18 s	132.699 ms (12.7%)
Agent	profiling	1.202 s	153.865 ms (14.7%)
Total	tracing	10.669 s	-
Total	appsec	10.742 s	73.162 ms (0.7%)
Total	iast	10.936 s	267.054 ms (2.5%)
Total	profiling	10.839 s	170.252 ms (1.6%)

gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~7c7286a04f, baseline=1.53.0-SNAPSHOT~5ed2f5961b

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.427 ms) : 0, 1427
crashtracking [candidate] (1.454 ms) : 0, 1454
BytebuddyAgent [baseline] (731.579 ms) : 0, 731579
BytebuddyAgent [candidate] (731.926 ms) : 0, 731926
GlobalTracer [baseline] (241.949 ms) : 0, 241949
GlobalTracer [candidate] (242.36 ms) : 0, 242360
AppSec [baseline] (30.034 ms) : 0, 30034
AppSec [candidate] (30.05 ms) : 0, 30050
Debugger [baseline] (6.037 ms) : 0, 6037
Debugger [candidate] (6.027 ms) : 0, 6027
Remote Config [baseline] (644.665 µs) : 0, 645
Remote Config [candidate] (653.383 µs) : 0, 653
Telemetry [baseline] (11.871 ms) : 0, 11871
Telemetry [candidate] (14.203 ms) : 0, 14203
section appsec
crashtracking [baseline] (1.43 ms) : 0, 1430
crashtracking [candidate] (1.426 ms) : 0, 1426
BytebuddyAgent [baseline] (751.745 ms) : 0, 751745
BytebuddyAgent [candidate] (752.355 ms) : 0, 752355
GlobalTracer [baseline] (234.443 ms) : 0, 234443
GlobalTracer [candidate] (234.804 ms) : 0, 234804
AppSec [baseline] (170.042 ms) : 0, 170042
AppSec [candidate] (169.312 ms) : 0, 169312
Debugger [baseline] (5.681 ms) : 0, 5681
Debugger [candidate] (7.141 ms) : 0, 7141
Remote Config [baseline] (616.282 µs) : 0, 616
Remote Config [candidate] (619.313 µs) : 0, 619
Telemetry [baseline] (9.118 ms) : 0, 9118
Telemetry [candidate] (8.311 ms) : 0, 8311
IAST [baseline] (23.427 ms) : 0, 23427
IAST [candidate] (23.507 ms) : 0, 23507
section iast
crashtracking [baseline] (1.45 ms) : 0, 1450
crashtracking [candidate] (1.438 ms) : 0, 1438
BytebuddyAgent [baseline] (855.333 ms) : 0, 855333
BytebuddyAgent [candidate] (852.23 ms) : 0, 852230
GlobalTracer [baseline] (232.368 ms) : 0, 232368
GlobalTracer [candidate] (232.93 ms) : 0, 232930
AppSec [baseline] (25.999 ms) : 0, 25999
AppSec [candidate] (26.344 ms) : 0, 26344
Debugger [baseline] (9.952 ms) : 0, 9952
Debugger [candidate] (9.28 ms) : 0, 9280
Remote Config [baseline] (573.661 µs) : 0, 574
Remote Config [candidate] (591.502 µs) : 0, 592
Telemetry [baseline] (8.213 ms) : 0, 8213
Telemetry [candidate] (8.383 ms) : 0, 8383
IAST [baseline] (27.545 ms) : 0, 27545
IAST [candidate] (28.127 ms) : 0, 28127
section profiling
crashtracking [baseline] (1.397 ms) : 0, 1397
crashtracking [candidate] (1.421 ms) : 0, 1421
BytebuddyAgent [baseline] (758.342 ms) : 0, 758342
BytebuddyAgent [candidate] (765.868 ms) : 0, 765868
GlobalTracer [baseline] (221.164 ms) : 0, 221164
GlobalTracer [candidate] (223.018 ms) : 0, 223018
AppSec [baseline] (29.734 ms) : 0, 29734
AppSec [candidate] (30.211 ms) : 0, 30211
Debugger [baseline] (6.244 ms) : 0, 6244
Debugger [candidate] (6.275 ms) : 0, 6275
Remote Config [baseline] (718.957 µs) : 0, 719
Remote Config [candidate] (696.976 µs) : 0, 697
Telemetry [baseline] (15.13 ms) : 0, 15130
Telemetry [candidate] (16.265 ms) : 0, 16265
ProfilingAgent [baseline] (109.15 ms) : 0, 109150
ProfilingAgent [candidate] (107.894 ms) : 0, 107894
Profiling [baseline] (109.801 ms) : 0, 109801
Profiling [candidate] (108.545 ms) : 0, 108545

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~7c7286a04f, baseline=1.53.0-SNAPSHOT~5ed2f5961b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.044 s) : 0, 1043590
Total [baseline] (8.591 s) : 0, 8590959
Agent [candidate] (1.046 s) : 0, 1045530
Total [candidate] (8.571 s) : 0, 8571425
section iast
Agent [baseline] (1.176 s) : 0, 1176342
Total [baseline] (9.293 s) : 0, 9293313
Agent [candidate] (1.184 s) : 0, 1183758
Total [candidate] (9.32 s) : 0, 9320464

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	iast	1.176 s	132.752 ms (12.7%)
Total	tracing	8.591 s	-
Total	iast	9.293 s	702.354 ms (8.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	iast	1.184 s	138.228 ms (13.2%)
Total	tracing	8.571 s	-
Total	iast	9.32 s	749.038 ms (8.7%)

gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~7c7286a04f, baseline=1.53.0-SNAPSHOT~5ed2f5961b

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.422 ms) : 0, 1422
crashtracking [candidate] (1.436 ms) : 0, 1436
BytebuddyAgent [baseline] (730.862 ms) : 0, 730862
BytebuddyAgent [candidate] (731.406 ms) : 0, 731406
GlobalTracer [baseline] (241.562 ms) : 0, 241562
GlobalTracer [candidate] (242.187 ms) : 0, 242187
AppSec [baseline] (30.023 ms) : 0, 30023
AppSec [candidate] (30.035 ms) : 0, 30035
Debugger [baseline] (6.013 ms) : 0, 6013
Debugger [candidate] (6.011 ms) : 0, 6011
Remote Config [baseline] (643.551 µs) : 0, 644
Remote Config [candidate] (645.772 µs) : 0, 646
Telemetry [baseline] (12.047 ms) : 0, 12047
Telemetry [candidate] (12.778 ms) : 0, 12778
section iast
crashtracking [baseline] (1.42 ms) : 0, 1420
crashtracking [candidate] (1.448 ms) : 0, 1448
BytebuddyAgent [baseline] (849.06 ms) : 0, 849060
BytebuddyAgent [candidate] (855.88 ms) : 0, 855880
GlobalTracer [baseline] (232.354 ms) : 0, 232354
GlobalTracer [candidate] (234.162 ms) : 0, 234162
AppSec [baseline] (27.589 ms) : 0, 27589
AppSec [candidate] (28.314 ms) : 0, 28314
Debugger [baseline] (7.512 ms) : 0, 7512
Debugger [candidate] (5.699 ms) : 0, 5699
Remote Config [baseline] (591.59 µs) : 0, 592
Remote Config [candidate] (578.717 µs) : 0, 579
Telemetry [baseline] (8.354 ms) : 0, 8354
Telemetry [candidate] (9.072 ms) : 0, 9072
IAST [baseline] (28.475 ms) : 0, 28475
IAST [candidate] (27.509 ms) : 0, 27509

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/junit-6
git_commit_date	1754661579	1754670324
git_commit_sha	`5ed2f59`	`7c7286a`
release_version	1.53.0-SNAPSHOT~5ed2f5961b	1.53.0-SNAPSHOT~7c7286a04f

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1754671863	1754671863
ci_job_id	1071882533	1071882533
ci_pipeline_id	73135578	73135578
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-p9fuxk1i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-p9fuxk1i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 11 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:appsec:high_load	worse [+2.014ms; +2.927ms] or [+4.353%; +6.326%]	unstable [-12.133op/s; +1.883op/s] or [-11.996%; +1.861%]	48.737ms	96.013op/s	46.267ms	101.138op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~7c7286a04f, baseline=1.53.0-SNAPSHOT~5ed2f5961b
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.328 ms) : 37023, 37633
.   : milestone, 37328,
appsec (46.267 ms) : 45854, 46680
.   : milestone, 46267,
code_origins (45.926 ms) : 45534, 46318
.   : milestone, 45926,
iast (45.114 ms) : 44718, 45511
.   : milestone, 45114,
profiling (48.338 ms) : 47845, 48830
.   : milestone, 48338,
tracing (44.778 ms) : 44392, 45164
.   : milestone, 44778,
section candidate
no_agent (36.803 ms) : 36508, 37098
.   : milestone, 36803,
appsec (48.737 ms) : 48302, 49173
.   : milestone, 48737,
code_origins (46.159 ms) : 45753, 46564
.   : milestone, 46159,
iast (46.209 ms) : 45807, 46610
.   : milestone, 46209,
profiling (49.189 ms) : 48720, 49658
.   : milestone, 49189,
tracing (45.845 ms) : 45454, 46235
.   : milestone, 45845,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.328 ms [37.023 ms, 37.633 ms]	-
appsec	46.267 ms [45.854 ms, 46.68 ms]	8.939 ms (23.9%)
code_origins	45.926 ms [45.534 ms, 46.318 ms]	8.598 ms (23.0%)
iast	45.114 ms [44.718 ms, 45.511 ms]	7.786 ms (20.9%)
profiling	48.338 ms [47.845 ms, 48.83 ms]	11.01 ms (29.5%)
tracing	44.778 ms [44.392 ms, 45.164 ms]	7.45 ms (20.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.803 ms [36.508 ms, 37.098 ms]	-
appsec	48.737 ms [48.302 ms, 49.173 ms]	11.934 ms (32.4%)
code_origins	46.159 ms [45.753 ms, 46.564 ms]	9.356 ms (25.4%)
iast	46.209 ms [45.807 ms, 46.61 ms]	9.406 ms (25.6%)
profiling	49.189 ms [48.72 ms, 49.658 ms]	12.386 ms (33.7%)
tracing	45.845 ms [45.454 ms, 46.235 ms]	9.041 ms (24.6%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~7c7286a04f, baseline=1.53.0-SNAPSHOT~5ed2f5961b
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.378 ms) : 4328, 4428
.   : milestone, 4378,
iast (9.427 ms) : 9273, 9580
.   : milestone, 9427,
iast_FULL (13.794 ms) : 13520, 14069
.   : milestone, 13794,
iast_GLOBAL (10.471 ms) : 10287, 10655
.   : milestone, 10471,
profiling (8.776 ms) : 8639, 8913
.   : milestone, 8776,
tracing (7.672 ms) : 7561, 7782
.   : milestone, 7672,
section candidate
no_agent (4.393 ms) : 4343, 4443
.   : milestone, 4393,
iast (9.393 ms) : 9242, 9545
.   : milestone, 9393,
iast_FULL (13.631 ms) : 13359, 13904
.   : milestone, 13631,
iast_GLOBAL (10.606 ms) : 10409, 10802
.   : milestone, 10606,
profiling (8.998 ms) : 8841, 9155
.   : milestone, 8998,
tracing (7.651 ms) : 7544, 7758
.   : milestone, 7651,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.378 ms [4.328 ms, 4.428 ms]	-
iast	9.427 ms [9.273 ms, 9.58 ms]	5.049 ms (115.3%)
iast_FULL	13.794 ms [13.52 ms, 14.069 ms]	9.416 ms (215.1%)
iast_GLOBAL	10.471 ms [10.287 ms, 10.655 ms]	6.093 ms (139.2%)
profiling	8.776 ms [8.639 ms, 8.913 ms]	4.398 ms (100.5%)
tracing	7.672 ms [7.561 ms, 7.782 ms]	3.294 ms (75.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.393 ms [4.343 ms, 4.443 ms]	-
iast	9.393 ms [9.242 ms, 9.545 ms]	5.0 ms (113.8%)
iast_FULL	13.631 ms [13.359 ms, 13.904 ms]	9.239 ms (210.3%)
iast_GLOBAL	10.606 ms [10.409 ms, 10.802 ms]	6.213 ms (141.4%)
profiling	8.998 ms [8.841 ms, 9.155 ms]	4.606 ms (104.8%)
tracing	7.651 ms [7.544 ms, 7.758 ms]	3.258 ms (74.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/junit-6
git_commit_date	1754661579	1754670324
git_commit_sha	`5ed2f59`	`7c7286a`
release_version	1.53.0-SNAPSHOT~5ed2f5961b	1.53.0-SNAPSHOT~7c7286a04f

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1754672359	1754672359
ci_job_id	1071882534	1071882534
ci_pipeline_id	73135578	73135578
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-fxsj6ppe 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-fxsj6ppe 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~7c7286a04f, baseline=1.53.0-SNAPSHOT~5ed2f5961b
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.495 s) : 15495000, 15495000
.   : milestone, 15495000,
appsec (14.844 s) : 14844000, 14844000
.   : milestone, 14844000,
iast (18.769 s) : 18769000, 18769000
.   : milestone, 18769000,
iast_GLOBAL (18.05 s) : 18050000, 18050000
.   : milestone, 18050000,
profiling (15.535 s) : 15535000, 15535000
.   : milestone, 15535000,
tracing (14.984 s) : 14984000, 14984000
.   : milestone, 14984000,
section candidate
no_agent (15.557 s) : 15557000, 15557000
.   : milestone, 15557000,
appsec (14.833 s) : 14833000, 14833000
.   : milestone, 14833000,
iast (18.502 s) : 18502000, 18502000
.   : milestone, 18502000,
iast_GLOBAL (18.136 s) : 18136000, 18136000
.   : milestone, 18136000,
profiling (15.278 s) : 15278000, 15278000
.   : milestone, 15278000,
tracing (15.198 s) : 15198000, 15198000
.   : milestone, 15198000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.495 s [15.495 s, 15.495 s]	-
appsec	14.844 s [14.844 s, 14.844 s]	-651.0 ms (-4.2%)
iast	18.769 s [18.769 s, 18.769 s]	3.274 s (21.1%)
iast_GLOBAL	18.05 s [18.05 s, 18.05 s]	2.555 s (16.5%)
profiling	15.535 s [15.535 s, 15.535 s]	40.0 ms (0.3%)
tracing	14.984 s [14.984 s, 14.984 s]	-511.0 ms (-3.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.557 s [15.557 s, 15.557 s]	-
appsec	14.833 s [14.833 s, 14.833 s]	-724.0 ms (-4.7%)
iast	18.502 s [18.502 s, 18.502 s]	2.945 s (18.9%)
iast_GLOBAL	18.136 s [18.136 s, 18.136 s]	2.579 s (16.6%)
profiling	15.278 s [15.278 s, 15.278 s]	-279.0 ms (-1.8%)
tracing	15.198 s [15.198 s, 15.198 s]	-359.0 ms (-2.3%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~7c7286a04f, baseline=1.53.0-SNAPSHOT~5ed2f5961b
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.484 ms) : 1472, 1495
.   : milestone, 1484,
appsec (2.495 ms) : 2442, 2548
.   : milestone, 2495,
iast (2.204 ms) : 2141, 2267
.   : milestone, 2204,
iast_GLOBAL (2.259 ms) : 2196, 2322
.   : milestone, 2259,
profiling (2.51 ms) : 2342, 2678
.   : milestone, 2510,
tracing (2.026 ms) : 1976, 2075
.   : milestone, 2026,
section candidate
no_agent (1.488 ms) : 1477, 1500
.   : milestone, 1488,
appsec (3.669 ms) : 3453, 3886
.   : milestone, 3669,
iast (2.212 ms) : 2150, 2275
.   : milestone, 2212,
iast_GLOBAL (2.258 ms) : 2195, 2321
.   : milestone, 2258,
profiling (2.051 ms) : 2001, 2101
.   : milestone, 2051,
tracing (2.05 ms) : 2001, 2100
.   : milestone, 2050,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.484 ms [1.472 ms, 1.495 ms]	-
appsec	2.495 ms [2.442 ms, 2.548 ms]	1.012 ms (68.2%)
iast	2.204 ms [2.141 ms, 2.267 ms]	720.217 µs (48.5%)
iast_GLOBAL	2.259 ms [2.196 ms, 2.322 ms]	775.353 µs (52.3%)
profiling	2.51 ms [2.342 ms, 2.678 ms]	1.027 ms (69.2%)
tracing	2.026 ms [1.976 ms, 2.075 ms]	542.17 µs (36.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.488 ms [1.477 ms, 1.5 ms]	-
appsec	3.669 ms [3.453 ms, 3.886 ms]	2.181 ms (146.5%)
iast	2.212 ms [2.15 ms, 2.275 ms]	724.184 µs (48.7%)
iast_GLOBAL	2.258 ms [2.195 ms, 2.321 ms]	769.646 µs (51.7%)
profiling	2.051 ms [2.001 ms, 2.101 ms]	562.899 µs (37.8%)
tracing	2.05 ms [2.001 ms, 2.1 ms]	562.112 µs (37.8%)

pr-commenter · 2025-08-08T13:26:16Z

Kafka / producer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/junit-6
git_commit_date	1754646927	1754670324
git_commit_sha	`aa86f15`	`7c7286a`

See matching parameters

	Baseline	Candidate
ci_job_date	1754671439	1754671439
ci_job_id	1071882537	1071882537
ci_pipeline_id	73135578	73135578
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce	unsure [-60422.403op/s; -6056.070op/s] or [-3.312%; -0.332%]
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce	same

pr-commenter · 2025-08-08T13:40:38Z

Kafka / consumer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/junit-6
git_commit_date	1754646927	1754657171
git_commit_sha	`aa86f15`	`78500f7`

See matching parameters

	Baseline	Candidate
ci_job_date	1754658602	1754658602
ci_job_id	1071265284	1071265284
ci_pipeline_id	73101776	73101776
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume	unsure [-11680.802op/s; -1003.753op/s] or [-3.931%; -0.338%]
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same

datadog-official · 2025-08-08T14:28:33Z

Code coverage: total 59.60%, base diff 2.28%, patch 100.00% (view details)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 7c7286a | Docs | Was this helpful? Give us feedback!}

AlexeyKuznetsov-DD · 2025-08-08T16:40:59Z

dd-java-agent/instrumentation-testing/src/test/groovy/context/FieldInjectionForkedTest.groovy

+    KeyClass                            | true  | true
+    UntransformableKeyClass             | false | false
+    ValidSerializableKeyClass           | true  | true
+    InvalidSerializableKeyClass         | true  | true
+    ValidInheritsSerializableKeyClass   | true  | true
+    InvalidInheritsSerializableKeyClass | true  | true


Is there a way to keep formatting?
here and couple of similar places

nikita-tkachenko-datadog added type: enhancement Enhancements and improvements comp: ci visibility Continuous Integration Visibility labels Jul 28, 2025

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/junit-6 branch 3 times, most recently from 6193b75 to c25d437 Compare August 1, 2025 14:41

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/junit-6 branch from f11f8af to 8034e05 Compare August 8, 2025 10:47

nikita-tkachenko-datadog added 3 commits August 8, 2025 12:49

Update AgentTestRunner to use JUnit5

3d0d50d

Fix iast and appsec test fixtures classpath

19c8a94

Fix agent-testing tests

4d6445f

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/junit-6 branch from 8034e05 to 4d6445f Compare August 8, 2025 10:49

nikita-tkachenko-datadog added 2 commits August 8, 2025 14:21

Fix IAST tests

deee7b8

Fix Jetty tests

78500f7

Fix IAST tests

1646230

nikita-tkachenko-datadog added 4 commits August 8, 2025 15:50

Fix servlet tests

4cc0b64

Fix log4j2 tests

ced6dd8

Fix Jetty tests

944303c

Fix log4j2 tests

c9f1105

nikita-tkachenko-datadog added 3 commits August 8, 2025 16:41

Removed a file committed accidentally

4998905

Minor cleanups

54b1788

Split CI Visibility test fixtures into a separate project

8d56d81

nikita-tkachenko-datadog added comp: testing Testing tag: no release notes Changes to exclude from release notes and removed comp: ci visibility Continuous Integration Visibility labels Aug 8, 2025

Fix Appsec tests

7c7286a

AlexeyKuznetsov-DD reviewed Aug 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Nikita tkachenko/junit 6 #9264

Nikita tkachenko/junit 6 #9264

Uh oh!

nikita-tkachenko-datadog commented Jul 28, 2025 •

edited

Loading

Uh oh!

AlexeyKuznetsov-DD commented Jul 28, 2025

Uh oh!

pr-commenter bot commented Jul 28, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Aug 8, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Aug 8, 2025

Uh oh!

datadog-official bot commented Aug 8, 2025 •

edited

Loading

Uh oh!

AlexeyKuznetsov-DD Aug 8, 2025

Uh oh!

Uh oh!

Nikita tkachenko/junit 6 #9264

Are you sure you want to change the base?

Nikita tkachenko/junit 6 #9264

Uh oh!

Conversation

nikita-tkachenko-datadog commented Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

AlexeyKuznetsov-DD commented Jul 28, 2025

Uh oh!

pr-commenter bot commented Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

pr-commenter bot commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Kafka / producer-benchmark

Parameters

Summary

Uh oh!

pr-commenter bot commented Aug 8, 2025

Kafka / consumer-benchmark

Parameters

Summary

Uh oh!

datadog-official bot commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexeyKuznetsov-DD Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nikita-tkachenko-datadog commented Jul 28, 2025 •

edited

Loading

pr-commenter bot commented Jul 28, 2025 •

edited

Loading

pr-commenter bot commented Aug 8, 2025 •

edited

Loading

datadog-official bot commented Aug 8, 2025 •

edited

Loading