Add testcase source file attribute to JUnit XML reports #8218

sarahchen6 · 2025-01-15T23:34:20Z

What Does This Do

This PR post-processes the XML results in collect_results.sh by backfilling source file information based on the XML report path and test class name.

If the result XML file is named after a specific test case, it sets the file attribute to the src directory that the test case was run from. Otherwise, we use grep to find the class file in this src directory and set the file attribute to the common path of the result(s).

Motivation

The goal of this PR is to add the source file attribute to the testcase nodes of the JUnit XML results. This will eventually allow us to assign codeowner data (see here) and log the test case source file for troubleshooting.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: https://datadoghq.atlassian.net/browse/LANGPLAT-147

pr-commenter · 2025-01-16T00:16:26Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-files-to-junit-reports-4
git_commit_date	1737470704	1737478543
git_commit_sha	`d5592c6`	`d44139d`
release_version	1.46.0-SNAPSHOT~d5592c6cbe	1.46.0-SNAPSHOT~d44139d825

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1737481018	1737481018
ci_job_id	774460007	774460007
ci_pipeline_id	53581152	53581152
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 5 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~d44139d825, baseline=1.46.0-SNAPSHOT~d5592c6cbe

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.056 s) : 0, 1056302
Total [baseline] (10.412 s) : 0, 10411573
Agent [candidate] (1.057 s) : 0, 1056829
Total [candidate] (10.468 s) : 0, 10468033
section appsec
Agent [baseline] (1.19 s) : 0, 1189859
Total [baseline] (10.687 s) : 0, 10687321
Agent [candidate] (1.189 s) : 0, 1189402
Total [candidate] (10.679 s) : 0, 10679041
section iast
Agent [baseline] (1.201 s) : 0, 1200537
Total [baseline] (10.992 s) : 0, 10991880
Agent [candidate] (1.187 s) : 0, 1187308
Total [candidate] (11.023 s) : 0, 11022592
section profiling
Agent [baseline] (1.264 s) : 0, 1263963
Total [baseline] (10.805 s) : 0, 10805242
Agent [candidate] (1.255 s) : 0, 1255340
Total [candidate] (10.802 s) : 0, 10802075

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	appsec	1.19 s	133.556 ms (12.6%)
Agent	iast	1.201 s	144.234 ms (13.7%)
Agent	profiling	1.264 s	207.661 ms (19.7%)
Total	tracing	10.412 s	-
Total	appsec	10.687 s	275.748 ms (2.6%)
Total	iast	10.992 s	580.307 ms (5.6%)
Total	profiling	10.805 s	393.669 ms (3.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.057 s	-
Agent	appsec	1.189 s	132.574 ms (12.5%)
Agent	iast	1.187 s	130.479 ms (12.3%)
Agent	profiling	1.255 s	198.511 ms (18.8%)
Total	tracing	10.468 s	-
Total	appsec	10.679 s	211.008 ms (2.0%)
Total	iast	11.023 s	554.559 ms (5.3%)
Total	profiling	10.802 s	334.042 ms (3.2%)

gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~d44139d825, baseline=1.46.0-SNAPSHOT~d5592c6cbe

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.366 ms) : 0, 714366
BytebuddyAgent [candidate] (715.165 ms) : 0, 715165
GlobalTracer [baseline] (255.888 ms) : 0, 255888
GlobalTracer [candidate] (255.592 ms) : 0, 255592
AppSec [baseline] (55.357 ms) : 0, 55357
AppSec [candidate] (56.853 ms) : 0, 56853
Remote Config [baseline] (743.666 µs) : 0, 744
Remote Config [candidate] (727.316 µs) : 0, 727
Telemetry [baseline] (14.944 ms) : 0, 14944
Telemetry [candidate] (13.42 ms) : 0, 13420
section appsec
BytebuddyAgent [baseline] (731.661 ms) : 0, 731661
BytebuddyAgent [candidate] (731.83 ms) : 0, 731830
GlobalTracer [baseline] (253.279 ms) : 0, 253279
GlobalTracer [candidate] (253.062 ms) : 0, 253062
AppSec [baseline] (171.414 ms) : 0, 171414
AppSec [candidate] (170.587 ms) : 0, 170587
Remote Config [baseline] (670.723 µs) : 0, 671
Remote Config [candidate] (659.391 µs) : 0, 659
Telemetry [baseline] (8.217 ms) : 0, 8217
Telemetry [candidate] (8.155 ms) : 0, 8155
IAST [baseline] (19.277 ms) : 0, 19277
IAST [candidate] (20.084 ms) : 0, 20084
section iast
BytebuddyAgent [baseline] (845.331 ms) : 0, 845331
BytebuddyAgent [candidate] (835.234 ms) : 0, 835234
GlobalTracer [baseline] (249.545 ms) : 0, 249545
GlobalTracer [candidate] (247.633 ms) : 0, 247633
AppSec [baseline] (58.7 ms) : 0, 58700
AppSec [candidate] (58.228 ms) : 0, 58228
Remote Config [baseline] (674.589 µs) : 0, 675
Remote Config [candidate] (673.197 µs) : 0, 673
Telemetry [baseline] (8.922 ms) : 0, 8922
Telemetry [candidate] (8.854 ms) : 0, 8854
IAST [baseline] (21.996 ms) : 0, 21996
IAST [candidate] (21.622 ms) : 0, 21622
section profiling
BytebuddyAgent [baseline] (709.216 ms) : 0, 709216
BytebuddyAgent [candidate] (704.729 ms) : 0, 704729
GlobalTracer [baseline] (351.402 ms) : 0, 351402
GlobalTracer [candidate] (350.372 ms) : 0, 350372
AppSec [baseline] (55.247 ms) : 0, 55247
AppSec [candidate] (53.942 ms) : 0, 53942
Remote Config [baseline] (720.415 µs) : 0, 720
Remote Config [candidate] (718.062 µs) : 0, 718
Telemetry [baseline] (8.854 ms) : 0, 8854
Telemetry [candidate] (8.742 ms) : 0, 8742
ProfilingAgent [baseline] (96.119 ms) : 0, 96119
ProfilingAgent [candidate] (94.718 ms) : 0, 94718
Profiling [baseline] (96.143 ms) : 0, 96143
Profiling [candidate] (94.743 ms) : 0, 94743

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~d44139d825, baseline=1.46.0-SNAPSHOT~d5592c6cbe

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1056692
Total [baseline] (8.657 s) : 0, 8657333
Agent [candidate] (1.064 s) : 0, 1063790
Total [candidate] (8.638 s) : 0, 8638422
section iast
Agent [baseline] (1.184 s) : 0, 1184236
Total [baseline] (9.183 s) : 0, 9182938
Agent [candidate] (1.187 s) : 0, 1186588
Total [candidate] (9.206 s) : 0, 9205707
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.184 s) : 0, 1184322
Total [baseline] (9.192 s) : 0, 9191824
Agent [candidate] (1.199 s) : 0, 1198834
Total [candidate] (9.193 s) : 0, 9193084
section iast_TELEMETRY_OFF
Agent [baseline] (1.188 s) : 0, 1187740
Total [baseline] (9.227 s) : 0, 9227418
Agent [candidate] (1.182 s) : 0, 1181951
Total [candidate] (9.197 s) : 0, 9196538

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.057 s	-
Agent	iast	1.184 s	127.544 ms (12.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.184 s	127.63 ms (12.1%)
Agent	iast_TELEMETRY_OFF	1.188 s	131.048 ms (12.4%)
Total	tracing	8.657 s	-
Total	iast	9.183 s	525.605 ms (6.1%)
Total	iast_HARDCODED_SECRET_DISABLED	9.192 s	534.491 ms (6.2%)
Total	iast_TELEMETRY_OFF	9.227 s	570.086 ms (6.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.064 s	-
Agent	iast	1.187 s	122.798 ms (11.5%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.199 s	135.044 ms (12.7%)
Agent	iast_TELEMETRY_OFF	1.182 s	118.161 ms (11.1%)
Total	tracing	8.638 s	-
Total	iast	9.206 s	567.285 ms (6.6%)
Total	iast_HARDCODED_SECRET_DISABLED	9.193 s	554.662 ms (6.4%)
Total	iast_TELEMETRY_OFF	9.197 s	558.116 ms (6.5%)

gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~d44139d825, baseline=1.46.0-SNAPSHOT~d5592c6cbe

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.848 ms) : 0, 714848
BytebuddyAgent [candidate] (720.924 ms) : 0, 720924
GlobalTracer [baseline] (255.741 ms) : 0, 255741
GlobalTracer [candidate] (258.165 ms) : 0, 258165
AppSec [baseline] (55.527 ms) : 0, 55527
AppSec [candidate] (55.823 ms) : 0, 55823
Remote Config [baseline] (730.913 µs) : 0, 731
Remote Config [candidate] (749.662 µs) : 0, 750
Telemetry [baseline] (14.788 ms) : 0, 14788
Telemetry [candidate] (12.965 ms) : 0, 12965
section iast
BytebuddyAgent [baseline] (833.163 ms) : 0, 833163
BytebuddyAgent [candidate] (835.089 ms) : 0, 835089
GlobalTracer [baseline] (246.761 ms) : 0, 246761
GlobalTracer [candidate] (247.474 ms) : 0, 247474
AppSec [baseline] (58.261 ms) : 0, 58261
AppSec [candidate] (58.017 ms) : 0, 58017
Remote Config [baseline] (663.887 µs) : 0, 664
Remote Config [candidate] (661.529 µs) : 0, 662
Telemetry [baseline] (8.759 ms) : 0, 8759
Telemetry [candidate] (8.806 ms) : 0, 8806
IAST [baseline] (21.582 ms) : 0, 21582
IAST [candidate] (21.49 ms) : 0, 21490
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (833.25 ms) : 0, 833250
BytebuddyAgent [candidate] (842.428 ms) : 0, 842428
GlobalTracer [baseline] (247.089 ms) : 0, 247089
GlobalTracer [candidate] (250.562 ms) : 0, 250562
AppSec [baseline] (58.037 ms) : 0, 58037
AppSec [candidate] (58.732 ms) : 0, 58732
Remote Config [baseline] (662.506 µs) : 0, 663
Remote Config [candidate] (698.206 µs) : 0, 698
Telemetry [baseline] (8.774 ms) : 0, 8774
Telemetry [candidate] (9.016 ms) : 0, 9016
IAST [baseline] (21.511 ms) : 0, 21511
IAST [candidate] (22.055 ms) : 0, 22055
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (835.758 ms) : 0, 835758
BytebuddyAgent [candidate] (831.018 ms) : 0, 831018
GlobalTracer [baseline] (248.176 ms) : 0, 248176
GlobalTracer [candidate] (247.579 ms) : 0, 247579
AppSec [baseline] (58.246 ms) : 0, 58246
AppSec [candidate] (58.063 ms) : 0, 58063
Remote Config [baseline] (653.692 µs) : 0, 654
Remote Config [candidate] (660.555 µs) : 0, 661
Telemetry [baseline] (8.688 ms) : 0, 8688
Telemetry [candidate] (8.645 ms) : 0, 8645
IAST [baseline] (21.039 ms) : 0, 21039
IAST [candidate] (20.92 ms) : 0, 20920

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-01-21T17:06:50	2025-01-21T17:13:56
git_branch	master	sarahchen6/add-files-to-junit-reports-4
git_commit_date	1737470704	1737478543
git_commit_sha	`d5592c6`	`d44139d`
release_version	1.46.0-SNAPSHOT~d5592c6cbe	1.46.0-SNAPSHOT~d44139d825
start_time	2025-01-21T17:06:36	2025-01-21T17:13:42

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1737479996	1737479996
ci_job_id	774460008	774460008
ci_pipeline_id	53581152	53581152
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~d44139d825, baseline=1.46.0-SNAPSHOT~d5592c6cbe
    dateFormat X
    axisFormat %s
section baseline
no_agent (385.424 µs) : 366, 405
.   : milestone, 385,
iast (513.184 µs) : 491, 535
.   : milestone, 513,
iast_FULL (751.907 µs) : 730, 774
.   : milestone, 752,
iast_GLOBAL (562.121 µs) : 540, 585
.   : milestone, 562,
iast_HARDCODED_SECRET_DISABLED (508.595 µs) : 486, 531
.   : milestone, 509,
iast_INACTIVE (462.75 µs) : 441, 484
.   : milestone, 463,
iast_TELEMETRY_OFF (497.641 µs) : 476, 520
.   : milestone, 498,
tracing (455.535 µs) : 434, 477
.   : milestone, 456,
section candidate
no_agent (385.56 µs) : 365, 406
.   : milestone, 386,
iast (517.713 µs) : 495, 540
.   : milestone, 518,
iast_FULL (748.823 µs) : 727, 771
.   : milestone, 749,
iast_GLOBAL (557.245 µs) : 535, 579
.   : milestone, 557,
iast_HARDCODED_SECRET_DISABLED (509.975 µs) : 488, 532
.   : milestone, 510,
iast_INACTIVE (462.895 µs) : 442, 484
.   : milestone, 463,
iast_TELEMETRY_OFF (499.803 µs) : 478, 522
.   : milestone, 500,
tracing (455.522 µs) : 435, 476
.   : milestone, 456,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	385.424 µs [365.799 µs, 405.05 µs]	-
iast	513.184 µs [491.309 µs, 535.058 µs]	127.759 µs (33.1%)
iast_FULL	751.907 µs [730.042 µs, 773.772 µs]	366.483 µs (95.1%)
iast_GLOBAL	562.121 µs [539.61 µs, 584.632 µs]	176.696 µs (45.8%)
iast_HARDCODED_SECRET_DISABLED	508.595 µs [486.386 µs, 530.804 µs]	123.171 µs (32.0%)
iast_INACTIVE	462.75 µs [441.118 µs, 484.382 µs]	77.325 µs (20.1%)
iast_TELEMETRY_OFF	497.641 µs [475.709 µs, 519.572 µs]	112.216 µs (29.1%)
tracing	455.535 µs [434.394 µs, 476.677 µs]	70.111 µs (18.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	385.56 µs [365.288 µs, 405.832 µs]	-
iast	517.713 µs [495.452 µs, 539.974 µs]	132.153 µs (34.3%)
iast_FULL	748.823 µs [726.923 µs, 770.724 µs]	363.263 µs (94.2%)
iast_GLOBAL	557.245 µs [535.396 µs, 579.094 µs]	171.685 µs (44.5%)
iast_HARDCODED_SECRET_DISABLED	509.975 µs [487.928 µs, 532.022 µs]	124.415 µs (32.3%)
iast_INACTIVE	462.895 µs [441.696 µs, 484.093 µs]	77.334 µs (20.1%)
iast_TELEMETRY_OFF	499.803 µs [478.049 µs, 521.557 µs]	114.243 µs (29.6%)
tracing	455.522 µs [434.623 µs, 476.421 µs]	69.962 µs (18.1%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~d44139d825, baseline=1.46.0-SNAPSHOT~d5592c6cbe
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.37 ms) : 1349, 1391
.   : milestone, 1370,
appsec (1.758 ms) : 1734, 1782
.   : milestone, 1758,
appsec_no_iast (1.76 ms) : 1736, 1784
.   : milestone, 1760,
iast (1.534 ms) : 1510, 1558
.   : milestone, 1534,
profiling (1.537 ms) : 1513, 1561
.   : milestone, 1537,
tracing (1.487 ms) : 1462, 1512
.   : milestone, 1487,
section candidate
no_agent (1.371 ms) : 1350, 1391
.   : milestone, 1371,
appsec (1.761 ms) : 1736, 1785
.   : milestone, 1761,
appsec_no_iast (1.777 ms) : 1752, 1802
.   : milestone, 1777,
iast (1.501 ms) : 1476, 1526
.   : milestone, 1501,
profiling (1.532 ms) : 1507, 1558
.   : milestone, 1532,
tracing (1.522 ms) : 1497, 1546
.   : milestone, 1522,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.37 ms [1.349 ms, 1.391 ms]	-
appsec	1.758 ms [1.734 ms, 1.782 ms]	388.619 µs (28.4%)
appsec_no_iast	1.76 ms [1.736 ms, 1.784 ms]	390.324 µs (28.5%)
iast	1.534 ms [1.51 ms, 1.558 ms]	164.715 µs (12.0%)
profiling	1.537 ms [1.513 ms, 1.561 ms]	167.243 µs (12.2%)
tracing	1.487 ms [1.462 ms, 1.512 ms]	117.429 µs (8.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.371 ms [1.35 ms, 1.391 ms]	-
appsec	1.761 ms [1.736 ms, 1.785 ms]	390.294 µs (28.5%)
appsec_no_iast	1.777 ms [1.752 ms, 1.802 ms]	406.588 µs (29.7%)
iast	1.501 ms [1.476 ms, 1.526 ms]	130.549 µs (9.5%)
profiling	1.532 ms [1.507 ms, 1.558 ms]	161.861 µs (11.8%)
tracing	1.522 ms [1.497 ms, 1.546 ms]	151.258 µs (11.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-files-to-junit-reports-4
git_commit_date	1737470704	1737478543
git_commit_sha	`d5592c6`	`d44139d`
release_version	1.46.0-SNAPSHOT~d5592c6cbe	1.46.0-SNAPSHOT~d44139d825

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1737480567	1737480567
ci_job_id	774460009	774460009
ci_pipeline_id	53581152	53581152
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~d44139d825, baseline=1.46.0-SNAPSHOT~d5592c6cbe
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1457, 1480
.   : milestone, 1469,
appsec (2.369 ms) : 2326, 2412
.   : milestone, 2369,
iast (2.113 ms) : 2058, 2167
.   : milestone, 2113,
iast_GLOBAL (2.155 ms) : 2100, 2210
.   : milestone, 2155,
profiling (2.473 ms) : 2286, 2659
.   : milestone, 2473,
tracing (1.947 ms) : 1905, 1989
.   : milestone, 1947,
section candidate
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (2.377 ms) : 2334, 2420
.   : milestone, 2377,
iast (2.108 ms) : 2053, 2163
.   : milestone, 2108,
iast_GLOBAL (2.158 ms) : 2103, 2213
.   : milestone, 2158,
profiling (1.985 ms) : 1940, 2030
.   : milestone, 1985,
tracing (1.946 ms) : 1904, 1988
.   : milestone, 1946,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.457 ms, 1.48 ms]	-
appsec	2.369 ms [2.326 ms, 2.412 ms]	900.576 µs (61.3%)
iast	2.113 ms [2.058 ms, 2.167 ms]	643.947 µs (43.8%)
iast_GLOBAL	2.155 ms [2.1 ms, 2.21 ms]	686.233 µs (46.7%)
profiling	2.473 ms [2.286 ms, 2.659 ms]	1.004 ms (68.3%)
tracing	1.947 ms [1.905 ms, 1.989 ms]	477.959 µs (32.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.463 ms, 1.486 ms]	-
appsec	2.377 ms [2.334 ms, 2.42 ms]	902.502 µs (61.2%)
iast	2.108 ms [2.053 ms, 2.163 ms]	633.632 µs (43.0%)
iast_GLOBAL	2.158 ms [2.103 ms, 2.213 ms]	683.538 µs (46.4%)
profiling	1.985 ms [1.94 ms, 2.03 ms]	510.17 µs (34.6%)
tracing	1.946 ms [1.904 ms, 1.988 ms]	471.552 µs (32.0%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~d44139d825, baseline=1.46.0-SNAPSHOT~d5592c6cbe
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.295 s) : 15295000, 15295000
.   : milestone, 15295000,
appsec (14.966 s) : 14966000, 14966000
.   : milestone, 14966000,
iast (18.961 s) : 18961000, 18961000
.   : milestone, 18961000,
iast_GLOBAL (17.851 s) : 17851000, 17851000
.   : milestone, 17851000,
profiling (15.786 s) : 15786000, 15786000
.   : milestone, 15786000,
tracing (14.873 s) : 14873000, 14873000
.   : milestone, 14873000,
section candidate
no_agent (15.096 s) : 15096000, 15096000
.   : milestone, 15096000,
appsec (15.045 s) : 15045000, 15045000
.   : milestone, 15045000,
iast (18.401 s) : 18401000, 18401000
.   : milestone, 18401000,
iast_GLOBAL (17.797 s) : 17797000, 17797000
.   : milestone, 17797000,
profiling (15.06 s) : 15060000, 15060000
.   : milestone, 15060000,
tracing (14.881 s) : 14881000, 14881000
.   : milestone, 14881000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.295 s [15.295 s, 15.295 s]	-
appsec	14.966 s [14.966 s, 14.966 s]	-329.0 ms (-2.2%)
iast	18.961 s [18.961 s, 18.961 s]	3.666 s (24.0%)
iast_GLOBAL	17.851 s [17.851 s, 17.851 s]	2.556 s (16.7%)
profiling	15.786 s [15.786 s, 15.786 s]	491.0 ms (3.2%)
tracing	14.873 s [14.873 s, 14.873 s]	-422.0 ms (-2.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.096 s [15.096 s, 15.096 s]	-
appsec	15.045 s [15.045 s, 15.045 s]	-51.0 ms (-0.3%)
iast	18.401 s [18.401 s, 18.401 s]	3.305 s (21.9%)
iast_GLOBAL	17.797 s [17.797 s, 17.797 s]	2.701 s (17.9%)
profiling	15.06 s [15.06 s, 15.06 s]	-36.0 ms (-0.2%)
tracing	14.881 s [14.881 s, 14.881 s]	-215.0 ms (-1.4%)

.circleci/collect_results.sh

sarahchen6 · 2025-01-17T15:29:44Z

.circleci/collect_results.sh

+function get_source_file () {
+  file_path="${RESULT_XML_FILE%%"/build"*}"
+  file_path="${file_path/#"$WORKSPACE_DIR"\//}/src"
+  if ! [[ $RESULT_XML_FILE == *"#"* ]]; then


If the XML file has "#" in the path, then the file name corresponds to the testcase name. In this case I left the source file as the src parent directory of the file that the test case was run from.

sarahchen6 · 2025-01-17T15:30:08Z

.circleci/collect_results.sh

  cp "$RESULT_XML_FILE" "$TEST_RESULTS_DIR/$AGGREGATED_FILE_NAME"
+  # Insert file attribute to testcase XML nodes
+  get_source_file
+  sed -i "/<testcase/ s|\(time=\"[^\"]*\"\)|\1 file=\"$file_path\"|g" "$TEST_RESULTS_DIR/$AGGREGATED_FILE_NAME"


This assumes that all testcases in the same XML result come from the same source file.

github-actions · 2025-01-17T15:54:03Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Remove the tag from the pull request title

If you need help, please check our contributing guidelines.

PerfectSlayer

Looks good as is. Let's give it a try and refine it if needed 👍

… their file name.

sarahchen6 force-pushed the sarahchen6/add-files-to-junit-reports-4 branch 8 times, most recently from 25c7e90 to 852f59e Compare January 17, 2025 03:41

sarahchen6 commented Jan 17, 2025

View reviewed changes

.circleci/collect_results.sh Outdated Show resolved Hide resolved

sarahchen6 commented Jan 17, 2025

View reviewed changes

.circleci/collect_results.sh Outdated Show resolved Hide resolved

sarahchen6 force-pushed the sarahchen6/add-files-to-junit-reports-4 branch 4 times, most recently from 22d8346 to a44012d Compare January 17, 2025 14:59

sarahchen6 commented Jan 17, 2025

View reviewed changes

sarahchen6 added type: documentation tag: no release notes Changes to exclude from release notes comp: ci visibility Continuous Integration Visibility comp: tooling Build & Tooling labels Jan 17, 2025

sarahchen6 changed the title ~~[WIP] Add file attribute to JUnit reports~~ [WIP] Add testcase source file attribute to JUnit XML reports Jan 17, 2025

sarahchen6 marked this pull request as ready for review January 17, 2025 15:53

sarahchen6 requested a review from a team as a code owner January 17, 2025 15:53

sarahchen6 requested a review from PerfectSlayer January 17, 2025 15:53

sarahchen6 changed the title ~~[WIP] Add testcase source file attribute to JUnit XML reports~~ Add testcase source file attribute to JUnit XML reports Jan 17, 2025

PerfectSlayer added type: enhancement Enhancements and improvements and removed comp: ci visibility Continuous Integration Visibility type: documentation labels Jan 17, 2025

sarahchen6 force-pushed the sarahchen6/add-files-to-junit-reports-4 branch from a44012d to f11fbf3 Compare January 21, 2025 14:16

PerfectSlayer approved these changes Jan 21, 2025

View reviewed changes

sarahchen6 added 8 commits January 21, 2025 11:55

Add script to insert source file attributes to XML file testcases.

be158e5

Change sed deliminator and add file quotes.

ec8e82b

Remove 'workspace/' from path.

271dc82

Rewrite file retrieval script to account for classes that don't match…

02776f6

… their file name.

Clean script.

779f69e

Add missing formatting for class name.

fcea1c6

Get common root of classes that are in multiple file paths.

42780e0

Lower case new variables.

d44139d

sarahchen6 force-pushed the sarahchen6/add-files-to-junit-reports-4 branch from f11fbf3 to d44139d Compare January 21, 2025 16:55

sarahchen6 added tag: no release notes Changes to exclude from release notes and removed tag: no release notes Changes to exclude from release notes labels Jan 21, 2025

sarahchen6 merged commit 1f71dbd into master Jan 21, 2025
175 checks passed

sarahchen6 deleted the sarahchen6/add-files-to-junit-reports-4 branch January 21, 2025 19:23

github-actions bot added this to the 1.46.0 milestone Jan 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add testcase source file attribute to JUnit XML reports #8218

Add testcase source file attribute to JUnit XML reports #8218

Uh oh!

sarahchen6 commented Jan 15, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Jan 16, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

sarahchen6 Jan 17, 2025

Uh oh!

sarahchen6 Jan 17, 2025

Uh oh!

github-actions bot commented Jan 17, 2025 •

edited

Loading

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add testcase source file attribute to JUnit XML reports #8218

Add testcase source file attribute to JUnit XML reports #8218

Uh oh!

Conversation

sarahchen6 commented Jan 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Jan 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Uh oh!

sarahchen6 Jan 17, 2025

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Jan 17, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sarahchen6 commented Jan 15, 2025 •

edited

Loading

pr-commenter bot commented Jan 16, 2025 •

edited

Loading

github-actions bot commented Jan 17, 2025 •

edited

Loading