Add support for instrumenting the OpenAI Client #9284

nayeem-kamal · 2025-07-31T11:43:41Z

What Does This Do

Adds auto-instrumentation for the official openai java client.

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

datadog-datadog-prod-us1 · 2025-07-31T12:06:10Z

Code coverage: total 57.09%, patch 100.00% (view details)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 298f31f | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-07-31T12:18:01Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nayeem-kamal/openai
git_commit_date	1755014538	1755021907
git_commit_sha	`ee43e5f`	`298f31f`
release_version	1.53.0-SNAPSHOT~ee43e5f19c	1.52.0-SNAPSHOT~298f31f1de

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1755024070	1755024070
ci_job_id	1076993448	1076993448
ci_pipeline_id	73464051	73464051
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-m7c1v81x 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-m7c1v81x 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 13 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.52.0-SNAPSHOT~298f31f1de, baseline=1.53.0-SNAPSHOT~ee43e5f19c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1042589
Total [baseline] (10.713 s) : 0, 10713027
Agent [candidate] (1.056 s) : 0, 1055804
Total [candidate] (10.79 s) : 0, 10789977
section appsec
Agent [baseline] (1.221 s) : 0, 1221168
Total [baseline] (10.689 s) : 0, 10689457
Agent [candidate] (1.219 s) : 0, 1219088
Total [candidate] (10.762 s) : 0, 10762333
section iast
Agent [baseline] (1.176 s) : 0, 1175991
Total [baseline] (10.909 s) : 0, 10909244
Agent [candidate] (1.176 s) : 0, 1175874
Total [candidate] (10.907 s) : 0, 10907372
section profiling
Agent [baseline] (1.193 s) : 0, 1192698
Total [baseline] (10.802 s) : 0, 10801538
Agent [candidate] (1.198 s) : 0, 1198225
Total [candidate] (10.897 s) : 0, 10896662

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.043 s	-
Agent	appsec	1.221 s	178.578 ms (17.1%)
Agent	iast	1.176 s	133.401 ms (12.8%)
Agent	profiling	1.193 s	150.108 ms (14.4%)
Total	tracing	10.713 s	-
Total	appsec	10.689 s	-23.57 ms (-0.2%)
Total	iast	10.909 s	196.216 ms (1.8%)
Total	profiling	10.802 s	88.51 ms (0.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	appsec	1.219 s	163.283 ms (15.5%)
Agent	iast	1.176 s	120.07 ms (11.4%)
Agent	profiling	1.198 s	142.42 ms (13.5%)
Total	tracing	10.79 s	-
Total	appsec	10.762 s	-27.644 ms (-0.3%)
Total	iast	10.907 s	117.395 ms (1.1%)
Total	profiling	10.897 s	106.685 ms (1.0%)

gantt
    title petclinic - break down per module: candidate=1.52.0-SNAPSHOT~298f31f1de, baseline=1.53.0-SNAPSHOT~ee43e5f19c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.42 ms) : 0, 1420
crashtracking [candidate] (1.444 ms) : 0, 1444
BytebuddyAgent [baseline] (730.076 ms) : 0, 730076
BytebuddyAgent [candidate] (739.012 ms) : 0, 739012
GlobalTracer [baseline] (241.412 ms) : 0, 241412
GlobalTracer [candidate] (243.631 ms) : 0, 243631
AppSec [baseline] (30.02 ms) : 0, 30020
AppSec [candidate] (30.494 ms) : 0, 30494
Debugger [baseline] (6.003 ms) : 0, 6003
Debugger [candidate] (6.083 ms) : 0, 6083
Remote Config [baseline] (651.574 µs) : 0, 652
Remote Config [candidate] (659.068 µs) : 0, 659
Telemetry [baseline] (12.099 ms) : 0, 12099
Telemetry [candidate] (13.348 ms) : 0, 13348
section appsec
crashtracking [baseline] (1.43 ms) : 0, 1430
crashtracking [candidate] (1.42 ms) : 0, 1420
BytebuddyAgent [baseline] (755.597 ms) : 0, 755597
BytebuddyAgent [candidate] (753.357 ms) : 0, 753357
GlobalTracer [baseline] (234.18 ms) : 0, 234180
GlobalTracer [candidate] (234.271 ms) : 0, 234271
IAST [baseline] (23.634 ms) : 0, 23634
IAST [candidate] (23.634 ms) : 0, 23634
AppSec [baseline] (169.162 ms) : 0, 169162
AppSec [candidate] (170.13 ms) : 0, 170130
Debugger [baseline] (6.382 ms) : 0, 6382
Debugger [candidate] (6.42 ms) : 0, 6420
Remote Config [baseline] (620.282 µs) : 0, 620
Remote Config [candidate] (609.139 µs) : 0, 609
Telemetry [baseline] (9.124 ms) : 0, 9124
Telemetry [candidate] (8.23 ms) : 0, 8230
section iast
crashtracking [baseline] (1.427 ms) : 0, 1427
crashtracking [candidate] (1.426 ms) : 0, 1426
BytebuddyAgent [baseline] (849.406 ms) : 0, 849406
BytebuddyAgent [candidate] (848.885 ms) : 0, 848885
GlobalTracer [baseline] (232.088 ms) : 0, 232088
GlobalTracer [candidate] (232.376 ms) : 0, 232376
IAST [baseline] (27.476 ms) : 0, 27476
IAST [candidate] (30.669 ms) : 0, 30669
AppSec [baseline] (27.559 ms) : 0, 27559
AppSec [candidate] (26.224 ms) : 0, 26224
Debugger [baseline] (8.223 ms) : 0, 8223
Debugger [candidate] (6.611 ms) : 0, 6611
Remote Config [baseline] (581.944 µs) : 0, 582
Remote Config [candidate] (599.779 µs) : 0, 600
Telemetry [baseline] (8.211 ms) : 0, 8211
Telemetry [candidate] (8.153 ms) : 0, 8153
section profiling
crashtracking [baseline] (1.392 ms) : 0, 1392
crashtracking [candidate] (1.405 ms) : 0, 1405
BytebuddyAgent [baseline] (760.304 ms) : 0, 760304
BytebuddyAgent [candidate] (763.583 ms) : 0, 763583
GlobalTracer [baseline] (221.372 ms) : 0, 221372
GlobalTracer [candidate] (222.39 ms) : 0, 222390
AppSec [baseline] (29.75 ms) : 0, 29750
AppSec [candidate] (30.197 ms) : 0, 30197
Debugger [baseline] (6.316 ms) : 0, 6316
Debugger [candidate] (6.319 ms) : 0, 6319
Remote Config [baseline] (681.764 µs) : 0, 682
Remote Config [candidate] (680.087 µs) : 0, 680
Telemetry [baseline] (15.84 ms) : 0, 15840
Telemetry [candidate] (15.957 ms) : 0, 15957
ProfilingAgent [baseline] (107.65 ms) : 0, 107650
ProfilingAgent [candidate] (108.106 ms) : 0, 108106
Profiling [baseline] (108.305 ms) : 0, 108305
Profiling [candidate] (108.755 ms) : 0, 108755

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.52.0-SNAPSHOT~298f31f1de, baseline=1.53.0-SNAPSHOT~ee43e5f19c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.044 s) : 0, 1043513
Total [baseline] (8.565 s) : 0, 8564622
Agent [candidate] (1.052 s) : 0, 1052456
Total [candidate] (8.574 s) : 0, 8573934
section iast
Agent [baseline] (1.177 s) : 0, 1177332
Total [baseline] (9.374 s) : 0, 9373931
Agent [candidate] (1.173 s) : 0, 1173185
Total [candidate] (9.298 s) : 0, 9297826

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	iast	1.177 s	133.819 ms (12.8%)
Total	tracing	8.565 s	-
Total	iast	9.374 s	809.309 ms (9.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	iast	1.173 s	120.729 ms (11.5%)
Total	tracing	8.574 s	-
Total	iast	9.298 s	723.893 ms (8.4%)

gantt
    title insecure-bank - break down per module: candidate=1.52.0-SNAPSHOT~298f31f1de, baseline=1.53.0-SNAPSHOT~ee43e5f19c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.428 ms) : 0, 1428
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (730.672 ms) : 0, 730672
BytebuddyAgent [candidate] (737.921 ms) : 0, 737921
GlobalTracer [baseline] (241.15 ms) : 0, 241150
GlobalTracer [candidate] (242.371 ms) : 0, 242371
AppSec [baseline] (29.95 ms) : 0, 29950
AppSec [candidate] (30.226 ms) : 0, 30226
Debugger [baseline] (5.985 ms) : 0, 5985
Debugger [candidate] (6.059 ms) : 0, 6059
Remote Config [baseline] (648.188 µs) : 0, 648
Remote Config [candidate] (650.076 µs) : 0, 650
Telemetry [baseline] (12.787 ms) : 0, 12787
Telemetry [candidate] (12.718 ms) : 0, 12718
section iast
crashtracking [baseline] (1.439 ms) : 0, 1439
crashtracking [candidate] (1.424 ms) : 0, 1424
BytebuddyAgent [baseline] (849.707 ms) : 0, 849707
BytebuddyAgent [candidate] (847.496 ms) : 0, 847496
GlobalTracer [baseline] (233.234 ms) : 0, 233234
GlobalTracer [candidate] (230.892 ms) : 0, 230892
IAST [baseline] (30.368 ms) : 0, 30368
IAST [candidate] (26.665 ms) : 0, 26665
AppSec [baseline] (26.08 ms) : 0, 26080
AppSec [candidate] (30.199 ms) : 0, 30199
Debugger [baseline] (6.654 ms) : 0, 6654
Debugger [candidate] (6.723 ms) : 0, 6723
Remote Config [baseline] (595.797 µs) : 0, 596
Remote Config [candidate] (593.925 µs) : 0, 594
Telemetry [baseline] (8.371 ms) : 0, 8371
Telemetry [candidate] (8.205 ms) : 0, 8205

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nayeem-kamal/openai
git_commit_date	1755014538	1755021907
git_commit_sha	`ee43e5f`	`298f31f`
release_version	1.53.0-SNAPSHOT~ee43e5f19c	1.52.0-SNAPSHOT~298f31f1de

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1755023752	1755023752
ci_job_id	1076993449	1076993449
ci_pipeline_id	73464051	73464051
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-o9iqzbex 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-o9iqzbex 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 3 performance regressions! Performance is the same for 6 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:no_agent:high_load	better [-252.156µs; -144.944µs] or [-5.575%; -3.205%]	unstable [-66.112op/s; +157.049op/s] or [-6.511%; +15.468%]	4.324ms	1060.781op/s	4.523ms	1015.312op/s
scenario:load:insecure-bank:profiling:high_load	worse [+221.203µs; +540.036µs] or [+2.534%; +6.187%]	unstable [-86.721op/s; +43.221op/s] or [-16.320%; +8.134%]	9.110ms	509.625op/s	8.729ms	531.375op/s
scenario:load:petclinic:tracing:high_load	better [-1.673ms; -0.912ms] or [-3.843%; -2.094%]	unstable [-4.866op/s; +11.391op/s] or [-4.529%; +10.603%]	42.255ms	110.688op/s	43.547ms	107.425op/s
scenario:load:petclinic:no_agent:high_load	worse [+1.781ms; +2.423ms] or [+4.992%; +6.790%]	unstable [-16.726op/s; +2.251op/s] or [-12.768%; +1.718%]	37.782ms	123.763op/s	35.681ms	131.000op/s
scenario:load:petclinic:iast:high_load	worse [+2.668ms; +3.519ms] or [+6.160%; +8.126%]	unstable [-13.766op/s; +2.019op/s] or [-12.748%; +1.870%]	46.402ms	102.114op/s	43.308ms	107.987op/s
scenario:load:petclinic:appsec:high_load	better [-1.917ms; -1.031ms] or [-4.002%; -2.152%]	unstable [-4.549op/s; +10.699op/s] or [-4.656%; +10.950%]	46.425ms	100.775op/s	47.899ms	97.700op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.52.0-SNAPSHOT~298f31f1de, baseline=1.53.0-SNAPSHOT~ee43e5f19c
    dateFormat X
    axisFormat %s
section baseline
no_agent (35.681 ms) : 35391, 35970
.   : milestone, 35681,
appsec (47.899 ms) : 47478, 48319
.   : milestone, 47899,
code_origins (45.36 ms) : 44965, 45755
.   : milestone, 45360,
iast (43.308 ms) : 42929, 43688
.   : milestone, 43308,
profiling (49.672 ms) : 49194, 50149
.   : milestone, 49672,
tracing (43.547 ms) : 43192, 43903
.   : milestone, 43547,
section candidate
no_agent (37.782 ms) : 37476, 38089
.   : milestone, 37782,
appsec (46.425 ms) : 46022, 46827
.   : milestone, 46425,
code_origins (45.565 ms) : 45157, 45973
.   : milestone, 45565,
iast (46.402 ms) : 45991, 46813
.   : milestone, 46402,
profiling (48.993 ms) : 48533, 49453
.   : milestone, 48993,
tracing (42.255 ms) : 41902, 42607
.   : milestone, 42255,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	35.681 ms [35.391 ms, 35.97 ms]	-
appsec	47.899 ms [47.478 ms, 48.319 ms]	12.218 ms (34.2%)
code_origins	45.36 ms [44.965 ms, 45.755 ms]	9.68 ms (27.1%)
iast	43.308 ms [42.929 ms, 43.688 ms]	7.628 ms (21.4%)
profiling	49.672 ms [49.194 ms, 50.149 ms]	13.991 ms (39.2%)
tracing	43.547 ms [43.192 ms, 43.903 ms]	7.867 ms (22.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.782 ms [37.476 ms, 38.089 ms]	-
appsec	46.425 ms [46.022 ms, 46.827 ms]	8.642 ms (22.9%)
code_origins	45.565 ms [45.157 ms, 45.973 ms]	7.783 ms (20.6%)
iast	46.402 ms [45.991 ms, 46.813 ms]	8.619 ms (22.8%)
profiling	48.993 ms [48.533 ms, 49.453 ms]	11.211 ms (29.7%)
tracing	42.255 ms [41.902 ms, 42.607 ms]	4.472 ms (11.8%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.52.0-SNAPSHOT~298f31f1de, baseline=1.53.0-SNAPSHOT~ee43e5f19c
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.523 ms) : 4471, 4574
.   : milestone, 4523,
iast (9.719 ms) : 9563, 9875
.   : milestone, 9719,
iast_FULL (13.96 ms) : 13686, 14235
.   : milestone, 13960,
iast_GLOBAL (10.543 ms) : 10356, 10730
.   : milestone, 10543,
profiling (8.729 ms) : 8586, 8872
.   : milestone, 8729,
tracing (7.749 ms) : 7628, 7870
.   : milestone, 7749,
section candidate
no_agent (4.324 ms) : 4276, 4372
.   : milestone, 4324,
iast (9.549 ms) : 9383, 9715
.   : milestone, 9549,
iast_FULL (14.158 ms) : 13878, 14438
.   : milestone, 14158,
iast_GLOBAL (10.28 ms) : 10086, 10474
.   : milestone, 10280,
profiling (9.11 ms) : 8957, 9263
.   : milestone, 9110,
tracing (7.566 ms) : 7455, 7678
.   : milestone, 7566,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.523 ms [4.471 ms, 4.574 ms]	-
iast	9.719 ms [9.563 ms, 9.875 ms]	5.197 ms (114.9%)
iast_FULL	13.96 ms [13.686 ms, 14.235 ms]	9.438 ms (208.7%)
iast_GLOBAL	10.543 ms [10.356 ms, 10.73 ms]	6.02 ms (133.1%)
profiling	8.729 ms [8.586 ms, 8.872 ms]	4.206 ms (93.0%)
tracing	7.749 ms [7.628 ms, 7.87 ms]	3.226 ms (71.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.324 ms [4.276 ms, 4.372 ms]	-
iast	9.549 ms [9.383 ms, 9.715 ms]	5.225 ms (120.8%)
iast_FULL	14.158 ms [13.878 ms, 14.438 ms]	9.834 ms (227.4%)
iast_GLOBAL	10.28 ms [10.086 ms, 10.474 ms]	5.956 ms (137.7%)
profiling	9.11 ms [8.957 ms, 9.263 ms]	4.786 ms (110.7%)
tracing	7.566 ms [7.455 ms, 7.678 ms]	3.242 ms (75.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nayeem-kamal/openai
git_commit_date	1755014538	1755021907
git_commit_sha	`ee43e5f`	`298f31f`
release_version	1.53.0-SNAPSHOT~ee43e5f19c	1.52.0-SNAPSHOT~298f31f1de

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1755024272	1755024272
ci_job_id	1076993450	1076993450
ci_pipeline_id	73464051	73464051
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-4tcnbvd2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-4tcnbvd2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:dacapo:tomcat:appsec	better [-1.314ms; -0.981ms] or [-36.309%; -27.109%]	2.471ms	3.618ms

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.52.0-SNAPSHOT~298f31f1de, baseline=1.53.0-SNAPSHOT~ee43e5f19c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.478 ms) : 1466, 1489
.   : milestone, 1478,
appsec (3.618 ms) : 3406, 3830
.   : milestone, 3618,
iast (2.213 ms) : 2150, 2276
.   : milestone, 2213,
iast_GLOBAL (2.254 ms) : 2190, 2317
.   : milestone, 2254,
profiling (2.046 ms) : 1995, 2096
.   : milestone, 2046,
tracing (2.024 ms) : 1976, 2073
.   : milestone, 2024,
section candidate
no_agent (1.478 ms) : 1466, 1489
.   : milestone, 1478,
appsec (2.471 ms) : 2418, 2523
.   : milestone, 2471,
iast (2.199 ms) : 2137, 2262
.   : milestone, 2199,
iast_GLOBAL (2.254 ms) : 2191, 2317
.   : milestone, 2254,
profiling (2.05 ms) : 2000, 2100
.   : milestone, 2050,
tracing (2.024 ms) : 1976, 2072
.   : milestone, 2024,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.466 ms, 1.489 ms]	-
appsec	3.618 ms [3.406 ms, 3.83 ms]	2.14 ms (144.8%)
iast	2.213 ms [2.15 ms, 2.276 ms]	734.882 µs (49.7%)
iast_GLOBAL	2.254 ms [2.19 ms, 2.317 ms]	775.798 µs (52.5%)
profiling	2.046 ms [1.995 ms, 2.096 ms]	567.628 µs (38.4%)
tracing	2.024 ms [1.976 ms, 2.073 ms]	546.554 µs (37.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.466 ms, 1.489 ms]	-
appsec	2.471 ms [2.418 ms, 2.523 ms]	993.162 µs (67.2%)
iast	2.199 ms [2.137 ms, 2.262 ms]	721.845 µs (48.9%)
iast_GLOBAL	2.254 ms [2.191 ms, 2.317 ms]	776.223 µs (52.5%)
profiling	2.05 ms [2.0 ms, 2.1 ms]	572.313 µs (38.7%)
tracing	2.024 ms [1.976 ms, 2.072 ms]	546.568 µs (37.0%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.52.0-SNAPSHOT~298f31f1de, baseline=1.53.0-SNAPSHOT~ee43e5f19c
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.932 s) : 14932000, 14932000
.   : milestone, 14932000,
appsec (14.893 s) : 14893000, 14893000
.   : milestone, 14893000,
iast (18.187 s) : 18187000, 18187000
.   : milestone, 18187000,
iast_GLOBAL (18.039 s) : 18039000, 18039000
.   : milestone, 18039000,
profiling (15.532 s) : 15532000, 15532000
.   : milestone, 15532000,
tracing (14.85 s) : 14850000, 14850000
.   : milestone, 14850000,
section candidate
no_agent (15.043 s) : 15043000, 15043000
.   : milestone, 15043000,
appsec (14.886 s) : 14886000, 14886000
.   : milestone, 14886000,
iast (18.97 s) : 18970000, 18970000
.   : milestone, 18970000,
iast_GLOBAL (18.045 s) : 18045000, 18045000
.   : milestone, 18045000,
profiling (15.682 s) : 15682000, 15682000
.   : milestone, 15682000,
tracing (14.988 s) : 14988000, 14988000
.   : milestone, 14988000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.932 s [14.932 s, 14.932 s]	-
appsec	14.893 s [14.893 s, 14.893 s]	-39.0 ms (-0.3%)
iast	18.187 s [18.187 s, 18.187 s]	3.255 s (21.8%)
iast_GLOBAL	18.039 s [18.039 s, 18.039 s]	3.107 s (20.8%)
profiling	15.532 s [15.532 s, 15.532 s]	600.0 ms (4.0%)
tracing	14.85 s [14.85 s, 14.85 s]	-82.0 ms (-0.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.043 s [15.043 s, 15.043 s]	-
appsec	14.886 s [14.886 s, 14.886 s]	-157.0 ms (-1.0%)
iast	18.97 s [18.97 s, 18.97 s]	3.927 s (26.1%)
iast_GLOBAL	18.045 s [18.045 s, 18.045 s]	3.002 s (20.0%)
profiling	15.682 s [15.682 s, 15.682 s]	639.0 ms (4.2%)
tracing	14.988 s [14.988 s, 14.988 s]	-55.0 ms (-0.4%)

PerfectSlayer

Left some comments to address before merging.

...trumentation/openai/src/main/java/datadog/trace/instrumentation/openai/OpenAIClientInfo.java

dd-java-agent/instrumentation/openai/build.gradle

PerfectSlayer · 2025-08-04T07:50:49Z

settings.gradle

@@ -430,6 +430,7 @@ include ':dd-java-agent:instrumentation:ognl-appsec'
 include ':dd-java-agent:instrumentation:opensearch'
 include ':dd-java-agent:instrumentation:opensearch:rest'
 include ':dd-java-agent:instrumentation:opensearch:transport'
+include ':dd-java-agent:instrumentation:openai'


🔨 issue: ‏Can you follow the layout described here?

🔨 issue: ‏ Not resolved. Made a suggestion to highlight the required change.

...src/main/java/datadog/trace/instrumentation/openai/ChatCompletionServiceInstrumentation.java

github-actions · 2025-08-04T11:15:18Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

amarziali · 2025-08-04T12:11:00Z

...i-client/src/main/java/datadog/trace/instrumentation/openaiclient/OpenAIClientDecorator.java

+          // Extract tool calls if present
+          int callIndex = 0;
+          for (ChatCompletionMessageToolCall call : toolCalls.get()) {
+            span.setTag(


nit: as a room for optimisation those tags can be perhaps cached. the second one is also sharing a big part of the first one

amarziali · 2025-08-04T12:13:29Z

...i-client/src/main/java/datadog/trace/instrumentation/openaiclient/OpenAIClientDecorator.java

+
+  public AgentScope startChatCompletionSpan(ChatCompletionCreateParams params) {
+    AgentSpan span = startSpan(OPENAI_REQUEST);
+    span.setTag("openai.request.endpoint", "/chat/completions");


nit: those could be stored as constants for a better reading

amarziali · 2025-08-04T12:17:37Z

...openai-client/src/main/java/datadog/trace/instrumentation/openaiclient/OpenAIClientInfo.java

+import com.openai.core.ClientOptions;
+import com.openai.credential.BearerTokenCredential;
+
+public class OpenAIClientInfo {


Is the information collected in this class used?

amarziali · 2025-08-04T12:18:22Z

...i-client/src/main/java/datadog/trace/instrumentation/openaiclient/OpenAIClientDecorator.java

+        llmObsSpan.setMetadata(responseMetadata);
+      }
+    } catch (Exception e) {
+      java.util.Map<String, Object> errorMetadata = new java.util.HashMap<>();


Map and HashMap should be imported

AlexeyKuznetsov-DD

Any tests for this change?

dd-java-agent/instrumentation/openai-client/gradle.lockfile

PerfectSlayer

Left comment about tracing instrumentation deactivation override, convention, possible dd-trace-ot, etc...

And where are the tests @AlexeyKuznetsov-DD asked for?

PerfectSlayer · 2025-08-06T14:35:14Z

settings.gradle.kts

@@ -459,6 +459,7 @@ include(
  ":dd-java-agent:instrumentation:okhttp-2",
  ":dd-java-agent:instrumentation:okhttp-3",
  ":dd-java-agent:instrumentation:ognl-appsec",
+  ":dd-java-agent:instrumentation:openai-client",


Suggested change

":dd-java-agent:instrumentation:openai-client",

":dd-java-agent:instrumentation:openai-java:openai-java-2.8",

PerfectSlayer · 2025-08-06T14:36:42Z

settings.gradle

@@ -430,6 +430,7 @@ include ':dd-java-agent:instrumentation:ognl-appsec'
 include ':dd-java-agent:instrumentation:opensearch'
 include ':dd-java-agent:instrumentation:opensearch:rest'
 include ':dd-java-agent:instrumentation:opensearch:transport'
+include ':dd-java-agent:instrumentation:openai'


🔨 issue: ‏ Not resolved. Made a suggestion to highlight the required change.

PerfectSlayer · 2025-08-06T14:40:02Z

...in/java/datadog/trace/instrumentation/openaiclient/ChatCompletionServiceInstrumentation.java

+public class ChatCompletionServiceInstrumentation extends InstrumenterModule.Tracing
+    implements Instrumenter.ForTypeHierarchy, Instrumenter.HasMethodAdvice {
+  public ChatCompletionServiceInstrumentation() {
+    super("openai-client", "openai-java", "openai-2.8");


🔨 issue: Conventions

Suggested change

super("openai-client", "openai-java", "openai-2.8");

super("openai-java", "openai-java-2.8", "openai-client");

PerfectSlayer · 2025-08-06T14:40:37Z

...in/java/datadog/trace/instrumentation/openaiclient/ChatCompletionServiceInstrumentation.java

+
+  @Override
+  protected boolean defaultEnabled() {
+    return InstrumenterConfig.get().isIntegrationEnabled(Collections.singleton("openai"), false);


🔨 issue: ‏This should not override tracing instrumentation deactivation.

PerfectSlayer · 2025-08-06T14:43:34Z

...c/main/java/datadog/trace/instrumentation/openaiclient/CompletionServiceInstrumentation.java

+public class CompletionServiceInstrumentation extends InstrumenterModule.Tracing
+    implements Instrumenter.ForTypeHierarchy, Instrumenter.HasMethodAdvice {
+  public CompletionServiceInstrumentation() {
+    super("openai-client", "openai-java", "openai-2.8");


🔨 issue: ‏Same

PerfectSlayer · 2025-08-06T14:44:51Z

...rc/main/java/datadog/trace/instrumentation/openaiclient/EmbeddingServiceInstrumentation.java

+
+  @Override
+  protected boolean defaultEnabled() {
+    return InstrumenterConfig.get().isIntegrationEnabled(Collections.singleton("openai"), false);


🔨 issue: ‏Same

PerfectSlayer · 2025-08-06T14:46:37Z

...nt/src/main/java/datadog/trace/instrumentation/openaiclient/OpenAIClientInstrumentation.java

+public class OpenAIClientInstrumentation extends InstrumenterModule.Tracing
+    implements Instrumenter.ForSingleType, Instrumenter.HasMethodAdvice {
+  public OpenAIClientInstrumentation() {
+    super("openai-client", "openai-java", "openai-2.8");


🔨 issue: ‏Same

PerfectSlayer · 2025-08-06T14:46:47Z

...nt/src/main/java/datadog/trace/instrumentation/openaiclient/OpenAIClientInstrumentation.java

+
+  @Override
+  protected boolean defaultEnabled() {
+    return InstrumenterConfig.get().isIntegrationEnabled(Collections.singleton("openai"), false);


🔨 issue: ‏Same

PerfectSlayer · 2025-08-06T14:50:24Z

dd-trace-core/build.gradle

@@ -72,6 +72,8 @@ dependencies {
  implementation project(':utils:socket-utils')
  // for span exception debugging
  compileOnly project(':dd-java-agent:agent-debugger:debugger-bootstrap')
+  // for llmobs
+  compileOnly project(':dd-java-agent:agent-llmobs')


🔨 issue: Will this work with dd-trace-ot?
Product should not be added here

PerfectSlayer · 2025-08-06T14:51:04Z

dd-java-agent/agent-llmobs/src/main/java/datadog/trace/llmobs/LLMObsSystem.java

@@ -22,7 +22,7 @@ public class LLMObsSystem {

  private static final String CUSTOM_MODEL_VAL = "custom";

-  public static void start(Instrumentation inst, SharedCommunicationObjects sco) {
+  public static void start(@Nullable Instrumentation inst, SharedCommunicationObjects sco) {


🎯 suggestion: ‏If this arg is always null and unused, why keeping it in the first place?

nayeem-kamal marked this pull request as ready for review August 4, 2025 04:23

nayeem-kamal requested review from a team as code owners August 4, 2025 04:23

nayeem-kamal requested review from amarziali and jordan-wong and removed request for amarziali August 4, 2025 04:23

jordan-wong approved these changes Aug 4, 2025

View reviewed changes

PerfectSlayer requested changes Aug 4, 2025

View reviewed changes

nayeem-kamal requested a review from a team as a code owner August 4, 2025 09:41

nayeem-kamal changed the title ~~OpenAI integration~~ Add support for instrumenting the OpenAI Client Aug 4, 2025

nayeem-kamal added type: feature request comp: mlobs ML Observability (LLMObs) labels Aug 4, 2025

nayeem-kamal requested a review from PerfectSlayer August 4, 2025 11:56

amarziali reviewed Aug 4, 2025

View reviewed changes

nayeem-kamal added 4 commits August 4, 2025 09:24

added apm openai instrumentation

378d442

added chatcompletion llmobs spans

42784e2

address PR comments

f620b66

resolve settings merge conflict

93a22fe

gary-huang force-pushed the nayeem-kamal/openai branch from f810778 to 93a22fe Compare August 4, 2025 13:29

gary-huang approved these changes Aug 4, 2025

View reviewed changes

import maps

ce288b1

gary-huang force-pushed the nayeem-kamal/openai branch from 98fe251 to ce288b1 Compare August 4, 2025 13:40

gary-huang enabled auto-merge (squash) August 4, 2025 13:44

AlexeyKuznetsov-DD requested changes Aug 4, 2025

View reviewed changes

dd-java-agent/instrumentation/openai-client/gradle.lockfile Outdated Show resolved Hide resolved

AlexeyKuznetsov-DD approved these changes Aug 4, 2025

View reviewed changes

PerfectSlayer requested changes Aug 6, 2025

View reviewed changes

nayeem-kamal added 2 commits August 12, 2025 10:11

addressed pr comments

0066c5d

address pr comments

298f31f

nayeem-kamal requested a review from PerfectSlayer August 13, 2025 04:52

	":dd-java-agent:instrumentation:openai-client",
	":dd-java-agent:instrumentation:openai-java:openai-java-2.8",

	super("openai-client", "openai-java", "openai-2.8");
	super("openai-java", "openai-java-2.8", "openai-client");

Add support for instrumenting the OpenAI Client #9284

Are you sure you want to change the base?

Add support for instrumenting the OpenAI Client #9284

Conversation

nayeem-kamal commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-datadog-prod-us1 bot commented Jul 31, 2025 • edited by datadog-official bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Aug 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlexeyKuznetsov-DD left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

PerfectSlayer left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nayeem-kamal commented Jul 31, 2025 •

edited

Loading

datadog-datadog-prod-us1 bot commented Jul 31, 2025 •

edited by datadog-official bot

Loading

pr-commenter bot commented Jul 31, 2025 •

edited

Loading

PerfectSlayer left a comment •

edited

Loading