Disable Debugger tracer smoke tests for j9 by jpbempel · Pull Request #8957 · DataDog/dd-trace-java

jpbempel · 2025-06-11T06:56:34Z

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-06-11T07:33:04Z

Benchmarks

Startup

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-06-11T07:07:03	2025-06-11T07:18:35
git_branch	master	jpbempel/disabled-smoke-tests-for-j9
git_commit_date	1749573424	1749624947
git_commit_sha	`3853c65`	`983c2db`
release_version	1.50.0-SNAPSHOT~3853c65296	1.50.0-SNAPSHOT~983c2dba36
start_time	2025-06-11T07:06:49	2025-06-11T07:18:09

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1749627107	1749627107
ci_job_id	976220753	976220753
ci_pipeline_id	67414032	67414032
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-x-xayjyg-project-304-concurrent-0-yagtxbv1 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-x-xayjyg-project-304-concurrent-0-yagtxbv1 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
thresholds_or_results	results	results
variant	iast	iast

Summary

Found 6 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 19 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:appsec_no_iast	better [-3.335ms; -3.235ms] or [-31.451%; -30.506%]	unstable [+177.311op/s; +231.570op/s] or [+37.901%; +49.499%]	7.318ms	672.269op/s	10.603ms	467.828op/s
scenario:load:petclinic:code_origins	better [-15.428ms; -14.825ms] or [-35.359%; -33.978%]	unstable [+43.110op/s; +73.318op/s] or [+39.629%; +67.397%]	28.505ms	167.000op/s	43.632ms	108.786op/s
scenario:load:petclinic:iast	better [-26.444ms; -25.891ms] or [-72.220%; -70.710%]	unstable [+321.683op/s; +360.631op/s] or [+240.853%; +270.014%]	10.449ms	474.717op/s	36.616ms	133.560op/s
scenario:load:petclinic:no_agent	better [-311.150µs; -262.566µs] or [-3.755%; -3.169%]	unsure [+7.458op/s; +38.823op/s] or [+1.249%; +6.503%]	7.999ms	620.155op/s	8.286ms	597.015op/s
scenario:load:petclinic:profiling	better [-560.026µs; -477.079µs] or [-65.189%; -55.533%]	unstable [-3747.810op/s; +11326.757op/s] or [-89.010%; +269.010%]	340.532µs	8000.000op/s	859.085µs	4210.526op/s
scenario:load:petclinic:tracing	better [-560.195µs; -476.560µs] or [-5.925%; -5.040%]	unstable [+2.092op/s; +55.605op/s] or [+0.400%; +10.634%]	8.937ms	551.724op/s	9.455ms	522.876op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~983c2dba36, baseline=1.50.0-SNAPSHOT~3853c65296
    dateFormat X
    axisFormat %s
section baseline
no_agent (8.286 ms) : 8263, 8309
.   : milestone, 8286,
appsec (150.766 µs) : 128, 174
.   : milestone, 151,
appsec_no_iast (10.603 ms) : 10543, 10663
.   : milestone, 10603,
code_origins (43.632 ms) : 43251, 44013
.   : milestone, 43632,
iast (36.616 ms) : 36256, 36977
.   : milestone, 36616,
profiling (859.085 µs) : 812, 906
.   : milestone, 859,
tracing (9.455 ms) : 9412, 9498
.   : milestone, 9455,
section candidate
no_agent (7.999 ms) : 7977, 8022
.   : milestone, 7999,
appsec (8.444 ms) : 8417, 8472
.   : milestone, 8444,
appsec_no_iast (7.318 ms) : 7292, 7345
.   : milestone, 7318,
code_origins (28.505 ms) : 28396, 28615
.   : milestone, 28505,
iast (10.449 ms) : 10405, 10492
.   : milestone, 10449,
profiling (340.532 µs) : 313, 368
.   : milestone, 341,
tracing (8.937 ms) : 8902, 8971
.   : milestone, 8937,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	8.286 ms [8.263 ms, 8.309 ms]	-
appsec	150.766 µs [127.672 µs, 173.861 µs]	-8.135 ms (-98.2%)
appsec_no_iast	10.603 ms [10.543 ms, 10.663 ms]	2.317 ms (28.0%)
code_origins	43.632 ms [43.251 ms, 44.013 ms]	35.346 ms (426.6%)
iast	36.616 ms [36.256 ms, 36.977 ms]	28.33 ms (341.9%)
profiling	859.085 µs [812.124 µs, 906.045 µs]	-7.427 ms (-89.6%)
tracing	9.455 ms [9.412 ms, 9.498 ms]	1.169 ms (14.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	7.999 ms [7.977 ms, 8.022 ms]	-
appsec	8.444 ms [8.417 ms, 8.472 ms]	445.171 µs (5.6%)
appsec_no_iast	7.318 ms [7.292 ms, 7.345 ms]	-681.008 µs (-8.5%)
code_origins	28.505 ms [28.396 ms, 28.615 ms]	20.506 ms (256.3%)
iast	10.449 ms [10.405 ms, 10.492 ms]	2.449 ms (30.6%)
profiling	340.532 µs [312.863 µs, 368.201 µs]	-7.659 ms (-95.7%)
tracing	8.937 ms [8.902 ms, 8.971 ms]	937.495 µs (11.7%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	jpbempel/disabled-smoke-tests-for-j9
git_commit_date	1749573424	1749624947
git_commit_sha	`3853c65`	`983c2db`
release_version	1.50.0-SNAPSHOT~3853c65296	1.50.0-SNAPSHOT~983c2dba36

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1749626944	1749626944
ci_job_id	976220754	976220754
ci_pipeline_id	67414032	67414032
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-ytwrhy75-project-304-concurrent-1-hzshnin4 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-ytwrhy75-project-304-concurrent-1-hzshnin4 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~983c2dba36, baseline=1.50.0-SNAPSHOT~3853c65296
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.48 ms) : 1468, 1491
.   : milestone, 1480,
appsec (2.407 ms) : 2359, 2456
.   : milestone, 2407,
iast (2.194 ms) : 2133, 2255
.   : milestone, 2194,
iast_GLOBAL (2.242 ms) : 2180, 2303
.   : milestone, 2242,
profiling (2.051 ms) : 2000, 2102
.   : milestone, 2051,
tracing (2.019 ms) : 1971, 2067
.   : milestone, 2019,
section candidate
no_agent (1.479 ms) : 1467, 1490
.   : milestone, 1479,
appsec (2.409 ms) : 2360, 2457
.   : milestone, 2409,
iast (2.188 ms) : 2127, 2249
.   : milestone, 2188,
iast_GLOBAL (2.229 ms) : 2168, 2291
.   : milestone, 2229,
profiling (2.504 ms) : 2327, 2680
.   : milestone, 2504,
tracing (2.015 ms) : 1968, 2063
.   : milestone, 2015,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.48 ms [1.468 ms, 1.491 ms]	-
appsec	2.407 ms [2.359 ms, 2.456 ms]	927.643 µs (62.7%)
iast	2.194 ms [2.133 ms, 2.255 ms]	713.855 µs (48.2%)
iast_GLOBAL	2.242 ms [2.18 ms, 2.303 ms]	761.694 µs (51.5%)
profiling	2.051 ms [2.0 ms, 2.102 ms]	571.087 µs (38.6%)
tracing	2.019 ms [1.971 ms, 2.067 ms]	539.174 µs (36.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.479 ms [1.467 ms, 1.49 ms]	-
appsec	2.409 ms [2.36 ms, 2.457 ms]	929.911 µs (62.9%)
iast	2.188 ms [2.127 ms, 2.249 ms]	709.262 µs (48.0%)
iast_GLOBAL	2.229 ms [2.168 ms, 2.291 ms]	750.628 µs (50.8%)
profiling	2.504 ms [2.327 ms, 2.68 ms]	1.025 ms (69.3%)
tracing	2.015 ms [1.968 ms, 2.063 ms]	536.555 µs (36.3%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~983c2dba36, baseline=1.50.0-SNAPSHOT~3853c65296
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.91 s) : 14910000, 14910000
.   : milestone, 14910000,
appsec (14.705 s) : 14705000, 14705000
.   : milestone, 14705000,
iast (18.893 s) : 18893000, 18893000
.   : milestone, 18893000,
iast_GLOBAL (18.181 s) : 18181000, 18181000
.   : milestone, 18181000,
profiling (15.085 s) : 15085000, 15085000
.   : milestone, 15085000,
tracing (14.91 s) : 14910000, 14910000
.   : milestone, 14910000,
section candidate
no_agent (15.319 s) : 15319000, 15319000
.   : milestone, 15319000,
appsec (14.77 s) : 14770000, 14770000
.   : milestone, 14770000,
iast (18.913 s) : 18913000, 18913000
.   : milestone, 18913000,
iast_GLOBAL (18.055 s) : 18055000, 18055000
.   : milestone, 18055000,
profiling (15.248 s) : 15248000, 15248000
.   : milestone, 15248000,
tracing (14.819 s) : 14819000, 14819000
.   : milestone, 14819000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.91 s [14.91 s, 14.91 s]	-
appsec	14.705 s [14.705 s, 14.705 s]	-205.0 ms (-1.4%)
iast	18.893 s [18.893 s, 18.893 s]	3.983 s (26.7%)
iast_GLOBAL	18.181 s [18.181 s, 18.181 s]	3.271 s (21.9%)
profiling	15.085 s [15.085 s, 15.085 s]	175.0 ms (1.2%)
tracing	14.91 s [14.91 s, 14.91 s]	0.0 µs (0.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.319 s [15.319 s, 15.319 s]	-
appsec	14.77 s [14.77 s, 14.77 s]	-549.0 ms (-3.6%)
iast	18.913 s [18.913 s, 18.913 s]	3.594 s (23.5%)
iast_GLOBAL	18.055 s [18.055 s, 18.055 s]	2.736 s (17.9%)
profiling	15.248 s [15.248 s, 15.248 s]	-71.0 ms (-0.5%)
tracing	14.819 s [14.819 s, 14.819 s]	-500.0 ms (-3.3%)

Disable Debugger tracer smoke tests for j9

983c2db

jpbempel requested a review from a team as a code owner June 11, 2025 06:56

jpbempel requested review from ojung and removed request for a team June 11, 2025 06:56

jpbempel added type: bug Bug report and fix comp: testing Testing tag: no release notes Changes to exclude from release notes comp: debugger Dynamic Instrumentation labels Jun 11, 2025

PerfectSlayer approved these changes Jun 11, 2025

View reviewed changes

jpbempel merged commit 6dbac83 into master Jun 11, 2025
520 of 521 checks passed

jpbempel deleted the jpbempel/disabled-smoke-tests-for-j9 branch June 11, 2025 08:03

github-actions bot added this to the 1.50.0 milestone Jun 11, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Disable Debugger tracer smoke tests for j9#8957

Disable Debugger tracer smoke tests for j9#8957
jpbempel merged 1 commit intomasterfrom
jpbempel/disabled-smoke-tests-for-j9

jpbempel commented Jun 11, 2025

Uh oh!

pr-commenter bot commented Jun 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jpbempel commented Jun 11, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Jun 11, 2025

Benchmarks

Startup

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants