Add telemetry for the RUM injector #9267

sarahchen6 · 2025-07-28T21:10:33Z

What Does This Do

Add telemetry for the RUM injector. Gather injection metrics and tags, and send the data via StatsDClient. The common metrics are based on: https://github.com/DataDog/dd-go/blob/prod/trace/apps/tracer-telemetry-intake/telemetry-metrics/static/common_metrics.json

Motivation

Complete the RUM Injector requirements (doc)

Additional Notes

This telemetry implementation does not include the following metrics:

injection.initialization.failed: Initialization failures can be due to a multitude of reasons and are already covered by SSI telemetry
injection.installation: Installation metrics are related to web server installers and not captured by this RUM injector implementation
injection.installation.duration: Installation metrics are related to web server installers and not captured by this RUM injector implementation

It also does not include the injector_version tag for the injection.failed metric. This tag refers to the Rust injector and we do not implement that.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: https://datadoghq.atlassian.net/browse/LANGPLAT-679

pr-commenter · 2025-07-28T21:48:07Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/implement-rum-injector-telemetry
git_commit_date	1755621799	1755631510
git_commit_sha	`27cd7f4`	`e9095b5`
release_version	1.53.0-SNAPSHOT~27cd7f43d9	1.53.0-SNAPSHOT~e9095b558b

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1755633311	1755633311
ci_job_id	1088151223	1088151223
ci_pipeline_id	74169254	74169254
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ns7pu4gu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ns7pu4gu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 13 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.052 s) : 0, 1051571
Total [baseline] (8.668 s) : 0, 8667699
Agent [candidate] (1.051 s) : 0, 1051243
Total [candidate] (8.652 s) : 0, 8651882
section iast
Agent [baseline] (1.201 s) : 0, 1200998
Total [baseline] (9.395 s) : 0, 9394589
Agent [candidate] (1.182 s) : 0, 1182393
Total [candidate] (9.336 s) : 0, 9335893

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	iast	1.201 s	149.427 ms (14.2%)
Total	tracing	8.668 s	-
Total	iast	9.395 s	726.891 ms (8.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.051 s	-
Agent	iast	1.182 s	131.15 ms (12.5%)
Total	tracing	8.652 s	-
Total	iast	9.336 s	684.011 ms (7.9%)

gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.47 ms) : 0, 1470
crashtracking [candidate] (1.471 ms) : 0, 1471
BytebuddyAgent [baseline] (736.502 ms) : 0, 736502
BytebuddyAgent [candidate] (736.823 ms) : 0, 736823
GlobalTracer [baseline] (243.819 ms) : 0, 243819
GlobalTracer [candidate] (242.808 ms) : 0, 242808
AppSec [baseline] (30.271 ms) : 0, 30271
AppSec [candidate] (30.032 ms) : 0, 30032
Debugger [baseline] (6.11 ms) : 0, 6110
Debugger [candidate] (6.03 ms) : 0, 6030
Remote Config [baseline] (672.375 µs) : 0, 672
Remote Config [candidate] (660.973 µs) : 0, 661
Telemetry [baseline] (10.912 ms) : 0, 10912
Telemetry [candidate] (12.276 ms) : 0, 12276
section iast
crashtracking [baseline] (1.492 ms) : 0, 1492
crashtracking [candidate] (1.457 ms) : 0, 1457
BytebuddyAgent [baseline] (868.458 ms) : 0, 868458
BytebuddyAgent [candidate] (853.106 ms) : 0, 853106
GlobalTracer [baseline] (236.069 ms) : 0, 236069
GlobalTracer [candidate] (233.957 ms) : 0, 233957
AppSec [baseline] (27.895 ms) : 0, 27895
AppSec [candidate] (27.834 ms) : 0, 27834
Debugger [baseline] (6.664 ms) : 0, 6664
Debugger [candidate] (5.799 ms) : 0, 5799
Remote Config [baseline] (606.786 µs) : 0, 607
Remote Config [candidate] (617.742 µs) : 0, 618
Telemetry [baseline] (8.463 ms) : 0, 8463
Telemetry [candidate] (8.489 ms) : 0, 8489
IAST [baseline] (30.024 ms) : 0, 30024
IAST [candidate] (30.094 ms) : 0, 30094

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.05 s) : 0, 1049804
Total [baseline] (10.706 s) : 0, 10706013
Agent [candidate] (1.047 s) : 0, 1047239
Total [candidate] (10.791 s) : 0, 10790805
section appsec
Agent [baseline] (1.231 s) : 0, 1231180
Total [baseline] (10.9 s) : 0, 10899853
Agent [candidate] (1.233 s) : 0, 1233491
Total [candidate] (10.868 s) : 0, 10868119
section iast
Agent [baseline] (1.192 s) : 0, 1192214
Total [baseline] (10.956 s) : 0, 10956312
Agent [candidate] (1.191 s) : 0, 1191286
Total [candidate] (10.985 s) : 0, 10985124
section profiling
Agent [baseline] (1.21 s) : 0, 1209699
Total [baseline] (11.134 s) : 0, 11133911
Agent [candidate] (1.197 s) : 0, 1196764
Total [candidate] (10.977 s) : 0, 10977017

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.05 s	-
Agent	appsec	1.231 s	181.376 ms (17.3%)
Agent	iast	1.192 s	142.41 ms (13.6%)
Agent	profiling	1.21 s	159.895 ms (15.2%)
Total	tracing	10.706 s	-
Total	appsec	10.9 s	193.84 ms (1.8%)
Total	iast	10.956 s	250.299 ms (2.3%)
Total	profiling	11.134 s	427.898 ms (4.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	appsec	1.233 s	186.253 ms (17.8%)
Agent	iast	1.191 s	144.047 ms (13.8%)
Agent	profiling	1.197 s	149.525 ms (14.3%)
Total	tracing	10.791 s	-
Total	appsec	10.868 s	77.314 ms (0.7%)
Total	iast	10.985 s	194.319 ms (1.8%)
Total	profiling	10.977 s	186.213 ms (1.7%)

gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.458 ms) : 0, 1458
crashtracking [candidate] (1.446 ms) : 0, 1446
BytebuddyAgent [baseline] (733.924 ms) : 0, 733924
BytebuddyAgent [candidate] (732.535 ms) : 0, 732535
GlobalTracer [baseline] (243.283 ms) : 0, 243283
GlobalTracer [candidate] (242.955 ms) : 0, 242955
AppSec [baseline] (30.107 ms) : 0, 30107
AppSec [candidate] (30.206 ms) : 0, 30206
Debugger [baseline] (6.086 ms) : 0, 6086
Debugger [candidate] (6.124 ms) : 0, 6124
Remote Config [baseline] (698.303 µs) : 0, 698
Remote Config [candidate] (662.761 µs) : 0, 663
Telemetry [baseline] (13.039 ms) : 0, 13039
Telemetry [candidate] (12.239 ms) : 0, 12239
section appsec
crashtracking [baseline] (1.463 ms) : 0, 1463
crashtracking [candidate] (1.458 ms) : 0, 1458
BytebuddyAgent [baseline] (760.865 ms) : 0, 760865
BytebuddyAgent [candidate] (762.453 ms) : 0, 762453
GlobalTracer [baseline] (237.166 ms) : 0, 237166
GlobalTracer [candidate] (237.752 ms) : 0, 237752
IAST [baseline] (23.951 ms) : 0, 23951
IAST [candidate] (23.846 ms) : 0, 23846
AppSec [baseline] (171.64 ms) : 0, 171640
AppSec [candidate] (169.655 ms) : 0, 169655
Debugger [baseline] (5.725 ms) : 0, 5725
Debugger [candidate] (7.208 ms) : 0, 7208
Remote Config [baseline] (628.851 µs) : 0, 629
Remote Config [candidate] (629.176 µs) : 0, 629
Telemetry [baseline] (8.429 ms) : 0, 8429
Telemetry [candidate] (9.192 ms) : 0, 9192
section iast
crashtracking [baseline] (1.466 ms) : 0, 1466
crashtracking [candidate] (1.464 ms) : 0, 1464
BytebuddyAgent [baseline] (860.967 ms) : 0, 860967
BytebuddyAgent [candidate] (859.95 ms) : 0, 859950
GlobalTracer [baseline] (235.212 ms) : 0, 235212
GlobalTracer [candidate] (235.233 ms) : 0, 235233
IAST [baseline] (29.613 ms) : 0, 29613
IAST [candidate] (29.965 ms) : 0, 29965
AppSec [baseline] (27.861 ms) : 0, 27861
AppSec [candidate] (27.744 ms) : 0, 27744
Debugger [baseline] (6.69 ms) : 0, 6690
Debugger [candidate] (6.633 ms) : 0, 6633
Remote Config [baseline] (615.418 µs) : 0, 615
Remote Config [candidate] (623.917 µs) : 0, 624
Telemetry [baseline] (8.502 ms) : 0, 8502
Telemetry [candidate] (8.426 ms) : 0, 8426
section profiling
crashtracking [baseline] (1.434 ms) : 0, 1434
crashtracking [candidate] (1.433 ms) : 0, 1433
BytebuddyAgent [baseline] (770.045 ms) : 0, 770045
BytebuddyAgent [candidate] (761.677 ms) : 0, 761677
GlobalTracer [baseline] (224.323 ms) : 0, 224323
GlobalTracer [candidate] (222.638 ms) : 0, 222638
AppSec [baseline] (30.375 ms) : 0, 30375
AppSec [candidate] (30.134 ms) : 0, 30134
Debugger [baseline] (6.32 ms) : 0, 6320
Debugger [candidate] (6.302 ms) : 0, 6302
Remote Config [baseline] (726.212 µs) : 0, 726
Remote Config [candidate] (727.627 µs) : 0, 728
Telemetry [baseline] (15.814 ms) : 0, 15814
Telemetry [candidate] (16.273 ms) : 0, 16273
ProfilingAgent [baseline] (110.585 ms) : 0, 110585
ProfilingAgent [candidate] (107.984 ms) : 0, 107984
Profiling [baseline] (111.255 ms) : 0, 111255
Profiling [candidate] (108.625 ms) : 0, 108625

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/implement-rum-injector-telemetry
git_commit_date	1755621799	1755631510
git_commit_sha	`27cd7f4`	`e9095b5`
release_version	1.53.0-SNAPSHOT~27cd7f43d9	1.53.0-SNAPSHOT~e9095b558b

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1755633093	1755633093
ci_job_id	1088151224	1088151224
ci_pipeline_id	74169254	74169254
cpu_model	Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-tb41jv1b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-tb41jv1b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 2 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:tracing:high_load	worse [+261.174µs; +525.754µs] or [+3.301%; +6.645%]	unstable [-96.489op/s; +41.177op/s] or [-16.475%; +7.031%]	8.305ms	558.000op/s	7.912ms	585.656op/s
scenario:load:petclinic:no_agent:high_load	better [-1.485ms; -0.814ms] or [-3.807%; -2.087%]	unstable [-6.127op/s; +10.215op/s] or [-5.044%; +8.410%]	37.867ms	123.513op/s	39.016ms	121.468op/s
scenario:load:petclinic:appsec:high_load	better [-2.116ms; -1.187ms] or [-4.313%; -2.418%]	unstable [-3.865op/s; +10.540op/s] or [-4.052%; +11.050%]	47.413ms	98.725op/s	49.065ms	95.388op/s
scenario:load:petclinic:iast:high_load	worse [+1.125ms; +2.002ms] or [+2.442%; +4.345%]	unstable [-10.515op/s; +3.890op/s] or [-10.353%; +3.830%]	47.647ms	98.250op/s	46.083ms	101.562op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
    dateFormat X
    axisFormat %s
section baseline
no_agent (39.016 ms) : 38693, 39340
.   : milestone, 39016,
appsec (49.065 ms) : 48620, 49509
.   : milestone, 49065,
code_origins (46.527 ms) : 46125, 46929
.   : milestone, 46527,
iast (46.083 ms) : 45685, 46482
.   : milestone, 46083,
profiling (50.047 ms) : 49598, 50496
.   : milestone, 50047,
tracing (45.073 ms) : 44681, 45465
.   : milestone, 45073,
section candidate
no_agent (37.867 ms) : 37567, 38166
.   : milestone, 37867,
appsec (47.413 ms) : 46994, 47832
.   : milestone, 47413,
code_origins (46.323 ms) : 45942, 46705
.   : milestone, 46323,
iast (47.647 ms) : 47231, 48063
.   : milestone, 47647,
profiling (49.301 ms) : 48813, 49789
.   : milestone, 49301,
tracing (45.974 ms) : 45570, 46379
.   : milestone, 45974,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	39.016 ms [38.693 ms, 39.34 ms]	-
appsec	49.065 ms [48.62 ms, 49.509 ms]	10.048 ms (25.8%)
code_origins	46.527 ms [46.125 ms, 46.929 ms]	7.51 ms (19.2%)
iast	46.083 ms [45.685 ms, 46.482 ms]	7.067 ms (18.1%)
profiling	50.047 ms [49.598 ms, 50.496 ms]	11.031 ms (28.3%)
tracing	45.073 ms [44.681 ms, 45.465 ms]	6.056 ms (15.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.867 ms [37.567 ms, 38.166 ms]	-
appsec	47.413 ms [46.994 ms, 47.832 ms]	9.547 ms (25.2%)
code_origins	46.323 ms [45.942 ms, 46.705 ms]	8.457 ms (22.3%)
iast	47.647 ms [47.231 ms, 48.063 ms]	9.78 ms (25.8%)
profiling	49.301 ms [48.813 ms, 49.789 ms]	11.434 ms (30.2%)
tracing	45.974 ms [45.57 ms, 46.379 ms]	8.108 ms (21.4%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.68 ms) : 4621, 4738
.   : milestone, 4680,
iast (9.987 ms) : 9815, 10159
.   : milestone, 9987,
iast_FULL (15.357 ms) : 15041, 15673
.   : milestone, 15357,
iast_GLOBAL (11.145 ms) : 10943, 11347
.   : milestone, 11145,
profiling (9.527 ms) : 9383, 9671
.   : milestone, 9527,
tracing (7.912 ms) : 7793, 8030
.   : milestone, 7912,
section candidate
no_agent (4.673 ms) : 4619, 4727
.   : milestone, 4673,
iast (9.691 ms) : 9534, 9848
.   : milestone, 9691,
iast_FULL (15.004 ms) : 14706, 15301
.   : milestone, 15004,
iast_GLOBAL (11.24 ms) : 11022, 11459
.   : milestone, 11240,
profiling (9.341 ms) : 9197, 9486
.   : milestone, 9341,
tracing (8.305 ms) : 8177, 8433
.   : milestone, 8305,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.68 ms [4.621 ms, 4.738 ms]	-
iast	9.987 ms [9.815 ms, 10.159 ms]	5.307 ms (113.4%)
iast_FULL	15.357 ms [15.041 ms, 15.673 ms]	10.677 ms (228.2%)
iast_GLOBAL	11.145 ms [10.943 ms, 11.347 ms]	6.466 ms (138.2%)
profiling	9.527 ms [9.383 ms, 9.671 ms]	4.847 ms (103.6%)
tracing	7.912 ms [7.793 ms, 8.03 ms]	3.232 ms (69.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.673 ms [4.619 ms, 4.727 ms]	-
iast	9.691 ms [9.534 ms, 9.848 ms]	5.018 ms (107.4%)
iast_FULL	15.004 ms [14.706 ms, 15.301 ms]	10.33 ms (221.1%)
iast_GLOBAL	11.24 ms [11.022 ms, 11.459 ms]	6.567 ms (140.5%)
profiling	9.341 ms [9.197 ms, 9.486 ms]	4.668 ms (99.9%)
tracing	8.305 ms [8.177 ms, 8.433 ms]	3.632 ms (77.7%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/implement-rum-injector-telemetry
git_commit_date	1755621799	1755631510
git_commit_sha	`27cd7f4`	`e9095b5`
release_version	1.53.0-SNAPSHOT~27cd7f43d9	1.53.0-SNAPSHOT~e9095b558b

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1755633556	1755633556
ci_job_id	1088151225	1088151225
ci_pipeline_id	74169254	74169254
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-azebvz3f 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-azebvz3f 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (3.659 ms) : 3441, 3876
.   : milestone, 3659,
iast (2.21 ms) : 2147, 2272
.   : milestone, 2210,
iast_GLOBAL (2.254 ms) : 2191, 2317
.   : milestone, 2254,
profiling (2.072 ms) : 2021, 2124
.   : milestone, 2072,
tracing (2.017 ms) : 1969, 2065
.   : milestone, 2017,
section candidate
no_agent (1.48 ms) : 1468, 1492
.   : milestone, 1480,
appsec (3.604 ms) : 3389, 3818
.   : milestone, 3604,
iast (2.213 ms) : 2150, 2277
.   : milestone, 2213,
iast_GLOBAL (2.251 ms) : 2188, 2314
.   : milestone, 2251,
profiling (2.056 ms) : 2004, 2108
.   : milestone, 2056,
tracing (2.035 ms) : 1986, 2084
.   : milestone, 2035,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.48 ms [1.469 ms, 1.492 ms]	-
appsec	3.659 ms [3.441 ms, 3.876 ms]	2.178 ms (147.1%)
iast	2.21 ms [2.147 ms, 2.272 ms]	729.184 µs (49.3%)
iast_GLOBAL	2.254 ms [2.191 ms, 2.317 ms]	773.609 µs (52.3%)
profiling	2.072 ms [2.021 ms, 2.124 ms]	591.975 µs (40.0%)
tracing	2.017 ms [1.969 ms, 2.065 ms]	536.705 µs (36.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.48 ms [1.468 ms, 1.492 ms]	-
appsec	3.604 ms [3.389 ms, 3.818 ms]	2.124 ms (143.5%)
iast	2.213 ms [2.15 ms, 2.277 ms]	733.504 µs (49.6%)
iast_GLOBAL	2.251 ms [2.188 ms, 2.314 ms]	771.259 µs (52.1%)
profiling	2.056 ms [2.004 ms, 2.108 ms]	576.381 µs (38.9%)
tracing	2.035 ms [1.986 ms, 2.084 ms]	555.003 µs (37.5%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.507 s) : 15507000, 15507000
.   : milestone, 15507000,
appsec (14.911 s) : 14911000, 14911000
.   : milestone, 14911000,
iast (18.67 s) : 18670000, 18670000
.   : milestone, 18670000,
iast_GLOBAL (18.245 s) : 18245000, 18245000
.   : milestone, 18245000,
profiling (15.131 s) : 15131000, 15131000
.   : milestone, 15131000,
tracing (14.869 s) : 14869000, 14869000
.   : milestone, 14869000,
section candidate
no_agent (15.494 s) : 15494000, 15494000
.   : milestone, 15494000,
appsec (14.821 s) : 14821000, 14821000
.   : milestone, 14821000,
iast (18.781 s) : 18781000, 18781000
.   : milestone, 18781000,
iast_GLOBAL (18.108 s) : 18108000, 18108000
.   : milestone, 18108000,
profiling (15.531 s) : 15531000, 15531000
.   : milestone, 15531000,
tracing (14.771 s) : 14771000, 14771000
.   : milestone, 14771000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.507 s [15.507 s, 15.507 s]	-
appsec	14.911 s [14.911 s, 14.911 s]	-596.0 ms (-3.8%)
iast	18.67 s [18.67 s, 18.67 s]	3.163 s (20.4%)
iast_GLOBAL	18.245 s [18.245 s, 18.245 s]	2.738 s (17.7%)
profiling	15.131 s [15.131 s, 15.131 s]	-376.0 ms (-2.4%)
tracing	14.869 s [14.869 s, 14.869 s]	-638.0 ms (-4.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.494 s [15.494 s, 15.494 s]	-
appsec	14.821 s [14.821 s, 14.821 s]	-673.0 ms (-4.3%)
iast	18.781 s [18.781 s, 18.781 s]	3.287 s (21.2%)
iast_GLOBAL	18.108 s [18.108 s, 18.108 s]	2.614 s (16.9%)
profiling	15.531 s [15.531 s, 15.531 s]	37.0 ms (0.2%)
tracing	14.771 s [14.771 s, 14.771 s]	-723.0 ms (-4.7%)

datadog-datadog-prod-us1 · 2025-07-31T15:08:33Z

Code coverage: total 57.51%, base diff 0.06%, patch 88.95% (view details)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: e9095b5 | Docs | Was this helpful? Give us feedback!}

PerfectSlayer

Very nice testing! 👏
I only few comments about minor improvements and readability.

...rc/test/groovy/datadog/trace/bootstrap/instrumentation/buffer/InjectingPipeWriterTest.groovy

.../src/main/java/datadog/trace/bootstrap/instrumentation/buffer/InjectingPipeOutputStream.java

...t/groovy/datadog/trace/bootstrap/instrumentation/buffer/InjectingPipeOutputStreamTest.groovy

...t/instrumentation/servlet/request-3/src/test/groovy/RumHttpServletResponseWrapperTest.groovy

internal-api/src/test/groovy/datadog/trace/api/rum/RumInjectorMetricsTest.groovy

internal-api/src/test/groovy/datadog/trace/api/rum/RumInjectorTest.groovy

PerfectSlayer

Thanks for all the follow up changes!
Nice work, especially around testing 👏

.../src/main/java/datadog/trace/bootstrap/instrumentation/buffer/InjectingPipeOutputStream.java

...st-3/src/main/java/datadog/trace/instrumentation/servlet3/RumHttpServletResponseWrapper.java

amarziali

It looks good. I left few comments or questions for improvement or clarification

internal-api/src/main/java/datadog/trace/api/rum/RumInjectorMetrics.java

sarahchen6 added type: feature request comp: metrics Metrics labels Jul 28, 2025

sarahchen6 force-pushed the sarahchen6/implement-rum-injector-telemetry branch 3 times, most recently from 1e2a4ed to 0687a05 Compare July 31, 2025 14:45

sarahchen6 force-pushed the sarahchen6/implement-rum-injector-telemetry branch from dbf1bee to 87c7127 Compare August 2, 2025 03:25

PerfectSlayer added comp: rum Realtime User Monitoring and removed comp: metrics Metrics labels Aug 8, 2025

sarahchen6 added 16 commits August 8, 2025 10:47

Add RumInjectorHealthMetrics

d19d27e

Add telemetry collector and methods to RumInjector

e01d1a9

Initialize health metrics and telemetry collector

8c71ef6

Get injectionsucceed count

6c01e10

Add comments

ceb3e11

Reorganize classes

92f0c54

Connect rum injector, telemetry collector, and statsdclient

c5c860f

Add tests

3d4ac53

Get and test metrics for injection failures and skips

3fd498d

Add Content Security Policy and HTTP response size telemetry

14c338b

Add injection duration telemetry

c5b5389

Fix some things

feb40be

Fix content-length retrieval and add test for injection timing

23e0455

Add injection initialization success telemetry

07db174

Fix CoreTracer compilation with InstrumenterConfig

a5352ff

Add tags to all metrics

2a1f676

sarahchen6 force-pushed the sarahchen6/implement-rum-injector-telemetry branch from 7bd7a48 to 2a1f676 Compare August 8, 2025 19:21

sarahchen6 added 3 commits August 8, 2025 16:42

Update InjectingPipeOutputStreamTest

b32874c

Tweaks

7a5cd68

Address jacoco coverage and injectingpipeoutstream interface updates

412a3f8

Address review comments

4b2f6b3

PerfectSlayer reviewed Aug 12, 2025

View reviewed changes

sarahchen6 added 2 commits August 12, 2025 14:35

Fix header retrieval

3e462d4

Add lots of improvements from review comments

b1ab871

PerfectSlayer approved these changes Aug 13, 2025

View reviewed changes

sarahchen6 changed the title ~~[WIP] Add telemetry for the RUM injector~~ Add telemetry for the RUM injector Aug 13, 2025

sarahchen6 marked this pull request as ready for review August 13, 2025 16:27

sarahchen6 requested review from a team as code owners August 13, 2025 16:27

sarahchen6 requested a review from amarziali August 13, 2025 16:27

sarahchen6 added 2 commits August 13, 2025 12:55

Fix constructors and address review comment

93bd0a4

Merge branch 'master' into sarahchen6/implement-rum-injector-telemetry

5a24d7d

AlexeyKuznetsov-DD reviewed Aug 13, 2025

View reviewed changes

.../src/main/java/datadog/trace/bootstrap/instrumentation/buffer/InjectingPipeOutputStream.java Outdated Show resolved Hide resolved

Clarify bytes written and address review comment

b20d6c4

AlexeyKuznetsov-DD approved these changes Aug 13, 2025

View reviewed changes

amarziali reviewed Aug 18, 2025

View reviewed changes

...st-3/src/main/java/datadog/trace/instrumentation/servlet3/RumHttpServletResponseWrapper.java Outdated Show resolved Hide resolved

amarziali reviewed Aug 18, 2025

View reviewed changes

...st-3/src/main/java/datadog/trace/instrumentation/servlet3/RumHttpServletResponseWrapper.java Outdated Show resolved Hide resolved

amarziali approved these changes Aug 18, 2025

View reviewed changes

sarahchen6 added 3 commits August 18, 2025 16:33

Use dynamic servlet version retrieval

f7607ba

Change injection timing logic

e6afc52

Clean up

6f2dc97

amarziali reviewed Aug 19, 2025

View reviewed changes

internal-api/src/main/java/datadog/trace/api/rum/RumInjectorMetrics.java Outdated Show resolved Hide resolved

sarahchen6 added 4 commits August 19, 2025 10:39

Use dynamic servlet version retrieval for tagging as well

32372c1

Address merge conflicts

821dc87

Add a telemetry check to HttpServerTest

150ab05

Clean up

e9095b5

amarziali reviewed Aug 20, 2025

View reviewed changes

internal-api/src/main/java/datadog/trace/api/rum/RumInjectorMetrics.java Show resolved Hide resolved

sarahchen6 merged commit 9bade09 into master Aug 20, 2025
666 of 667 checks passed

sarahchen6 deleted the sarahchen6/implement-rum-injector-telemetry branch August 20, 2025 12:41

github-actions bot added this to the 1.53.0 milestone Aug 20, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add telemetry for the RUM injector #9267

Add telemetry for the RUM injector #9267

Uh oh!

sarahchen6 commented Jul 28, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Jul 28, 2025 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 bot commented Jul 31, 2025 •

edited by datadog-official bot

Loading

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amarziali left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Add telemetry for the RUM injector #9267

Add telemetry for the RUM injector #9267

Uh oh!

Conversation

sarahchen6 commented Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

datadog-datadog-prod-us1 bot commented Jul 31, 2025 • edited by datadog-official bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amarziali left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sarahchen6 commented Jul 28, 2025 •

edited

Loading

pr-commenter bot commented Jul 28, 2025 •

edited

Loading

datadog-datadog-prod-us1 bot commented Jul 31, 2025 •

edited by datadog-official bot

Loading