feat(instrumentation-openai): add instrumentation of openai SDK #2941

anuraaga · 2025-07-17T09:23:36Z

Which problem is this PR solving?

openai SDK usage should have gen ai conventions but there is no instrumentation yet

Short description of the changes

Adds instrumentation targeting openai SDK 4 for chat and embeddings

Future PRs will add support for responses API and SDK 5

/cc @trentm if you can help review
/cc @lmolkova because I heard you are also interested in this instrumentation. Notably, I'd like this instrumentation to exist upstream so people can use it but need potential component owners I believe

This instrumentation is adapted from the one in https://github.com/elastic/elastic-otel-node/tree/main/packages/instrumentation-openai

anuraaga · 2025-07-18T01:49:54Z

packages/instrumentation-openai/test/openai.test.ts

+        })
+      ).rejects.toThrow(OpenAI.APIConnectionError);
+
+      // TODO: Figure out why it takes so long to get this span. trace.getTracerProvider()._delegate.forceFlush() didn't help.


Could use some help debugging this

codecov · 2025-07-18T01:52:20Z

Codecov Report

❌ Patch coverage is 87.96296% with 39 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.81%. Comparing base (cde6d4d) to head (6d77251).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...ages/instrumentation-openai/src/instrumentation.ts	90.90%	25 Missing ⚠️
packages/instrumentation-openai/src/utils.ts	53.33%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2941      +/-   ##
==========================================
- Coverage   89.86%   89.81%   -0.06%     
==========================================
  Files         189      192       +3     
  Lines        9314     9638     +324     
  Branches     1911     1988      +77     
==========================================
+ Hits         8370     8656     +286     
- Misses        944      982      +38

Files with missing lines	Coverage Δ
packages/instrumentation-openai/src/semconv.ts	`100.00% <100.00%> (ø)`
packages/instrumentation-openai/src/utils.ts	`53.33% <53.33%> (ø)`
...ages/instrumentation-openai/src/instrumentation.ts	`90.90% <90.90%> (ø)`

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…ry-js-contrib into openai-inst

nx.json

trentm

Thanks for starting this, Anuraag. This is just a quick start at looking. I haven't looked at src/ or test/ yet.

packages/instrumentation-openai/package.json

nx.json

…y-js-contrib into openai-inst

packages/instrumentation-openai/src/utils.ts

packages/instrumentation-openai/src/instrumentation.ts

seemk · 2025-07-26T09:04:46Z

packages/instrumentation-openai/src/instrumentation.ts

+              body.content = msg.content;
+            }
+          }
+          this.logger.emit({


If one has set up a global logging provider, is it possible to opt out of the logs emitted from this instrumentation?

@seemk I think one could do so by setting a NoopLoggerProvider on the instrumentation via instr.setLoggerProvider(noopLoggerProvider).

But are you asking for something different than that?

I haven't been following the GenAI semconv discussions of late. I vaguely recall that there was discussion of making it user-optional whether GenAI instrumentations used log-events or span attributes. Is that what you are referring to?

Yeah - from what I understand, the conventions imply that when logs are enabled, logs are always emitted - this is how it currently works in the bedrock instrumentation of this repo, as well as genai in python and java. If it seems like there should be more knobs, we'd need to raise changes to the semconv but for now this implements the current practice.

trentm

Another round of review, looking at the implementation this time. Looks good! Thanks for all the type improvements.

I haven't yet looked at the tests.

packages/instrumentation-openai/src/instrumentation.ts

packages/instrumentation-openai/src/semconv.ts

Co-authored-by: Trent Mick <[email protected]>

…ry-js-contrib into openai-inst

packages/instrumentation-openai/package.json

trentm

Tests look good to me (with some comments below). I didn't comb through all 3000 lines; about half of them.

packages/instrumentation-openai/src/instrumentation.ts

packages/instrumentation-openai/test/openai.test.ts

packages/instrumentation-openai/test/load-instrumentation.ts

packages/instrumentation-openai/test/openai.test.ts

trentm

A couple more things: test-all-versions, and openai@5 support.

packages/instrumentation-openai/package.json

packages/instrumentation-openai/src/instrumentation.ts

…y-js-contrib into openai-inst

trentm · 2025-08-01T15:44:18Z

This LGTM.

Not sure if you were going to wait for feedback from the MS folks that might help and possibly be component owners.

Before or when being taken out of draft, this PR will need to add one or more component owners to ".github/component_owners.yml". I could be one if needed, at least for starters.

trentm · 2025-08-01T22:28:03Z

packages/instrumentation-openai/src/instrumentation.ts

+const EVENT_GEN_AI_USER_MESSAGE = 'gen_ai.user.message';
+const EVENT_GEN_AI_ASSISTANT_MESSAGE = 'gen_ai.assistant.message';
+const EVENT_GEN_AI_TOOL_MESSAGE = 'gen_ai.tool.message';
+const EVENT_GEN_AI_CHOICE = 'gen_ai.choice';


FWIW, I've started a PR to eventually add EVENT_* exports from the semantic-conventions package.
open-telemetry/opentelemetry-js#5832

…y-js-contrib into openai-inst

anuraaga · 2025-08-07T03:11:22Z

Thanks @trentm - I have gone ahead and updated owners as if possible, it would be nice to avoid drift by merging an initial version. Will continue discussion to hopefully add more /cc @paulshealy1

trentm · 2025-08-19T17:11:08Z

I asked for possible additional component owners for this instrumentation on the GenAI SIG today: https://docs.google.com/document/d/1EKIeDgBGXQPGehUigIRLwAUpRGa7-1kXB736EaYuJ2M/edit?tab=t.0#heading=h.tath31lg83xm

Being a "component owner" means that you are listed for the particular instrumentation here: https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/.github/component_owners.yml
and that you will be tagged for feedback/review for issues and PRs related to this instrumentation.
The idea is that you are able to provide informed review and opinions on telemetry design for the target module (in this case for openai). I can handle any instrumentation details related to opentelemetry-js internals or general instrumentation issues.

No pressure, but there was possible interest expressed by @zhirafovod. (@seemk would you be interested in being a component owner for this instrumentation?)

As well, @singankit was possibly going to poke @paulshealy1 again internally.

Thanks.

seemk · 2025-08-19T17:51:04Z

I asked for possible additional component owners for this instrumentation on the GenAI SIG today: https://docs.google.com/document/d/1EKIeDgBGXQPGehUigIRLwAUpRGa7-1kXB736EaYuJ2M/edit?tab=t.0#heading=h.tath31lg83xm

Being a "component owner" means that you are listed for the particular instrumentation here: https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/.github/component_owners.yml and that you will be tagged for feedback/review for issues and PRs related to this instrumentation. The idea is that you are able to provide informed review and opinions on telemetry design for the target module (in this case for openai). I can handle any instrumentation details related to opentelemetry-js internals or general instrumentation issues.

No pressure, but there was possible interest expressed by @zhirafovod. (@seemk would you be interested in being a component owner for this instrumentation?)

As well, @singankit was possibly going to poke @paulshealy1 again internally.

Thanks.

I can help out, yeah

trentm

@anuraaga One suggested change (adding seemk as a component owner), then LGTM. Thanks!

.github/component_owners.yml

Co-authored-by: Trent Mick <[email protected]>

anuraaga · 2025-08-20T01:41:58Z

Thanks a lot @seemk and @trentm!

feat(instrumentation-openai): add instrumentation of openai SDK

dcfff66

anuraaga requested a review from a team as a code owner July 17, 2025 09:23

anuraaga marked this pull request as draft July 17, 2025 09:23

anuraaga added 3 commits July 17, 2025 18:26

Fix manifest

b58453e

Fix types

152c2aa

Merge branch 'main' into openai-inst

20e7bad

anuraaga commented Jul 18, 2025

View reviewed changes

anuraaga added 2 commits July 18, 2025 11:00

version:update

8284f96

Merge branch 'openai-inst' of https://github.com/anuraaga/opentelemet…

d023355

…ry-js-contrib into openai-inst

anuraaga commented Jul 18, 2025

View reviewed changes

nx.json Outdated Show resolved Hide resolved

trentm reviewed Jul 18, 2025

View reviewed changes

anuraaga added 3 commits July 19, 2025 10:26

Cleanups

c8e58c6

Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…

380b165

…y-js-contrib into openai-inst

Fix

22e806a

seemk reviewed Jul 26, 2025

View reviewed changes

trentm mentioned this pull request Jul 30, 2025

feat: Adding intial support for openai instrumentation for chat creation #2402

Closed

trentm reviewed Jul 30, 2025

View reviewed changes

packages/instrumentation-openai/src/instrumentation.ts Show resolved Hide resolved

packages/instrumentation-openai/src/instrumentation.ts Show resolved Hide resolved

packages/instrumentation-openai/src/semconv.ts Outdated Show resolved Hide resolved

anuraaga and others added 7 commits July 31, 2025 10:57

Update packages/instrumentation-openai/src/utils.ts

16d44e6

Co-authored-by: Trent Mick <[email protected]>

Fix debug

267be04

Merge branch 'openai-inst' of https://github.com/anuraaga/opentelemet…

44311d7

…ry-js-contrib into openai-inst

Comment

c6ac4ac

Regenerate semconv

a69be29

Format

52c26b5

Fix

3ee2ed6

trentm reviewed Jul 31, 2025

View reviewed changes

packages/instrumentation-openai/package.json Outdated Show resolved Hide resolved

trentm reviewed Jul 31, 2025

View reviewed changes

packages/instrumentation-openai/package.json Show resolved Hide resolved

packages/instrumentation-openai/src/instrumentation.ts Outdated Show resolved Hide resolved

anuraaga added 2 commits August 1, 2025 10:42

Cleanup

65c2a5f

git add

5b0e035

anuraaga added 4 commits August 1, 2025 10:46

README

cc1ada2

Match expect version

bb28e1a

Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…

6b90002

…y-js-contrib into openai-inst

manage to restore package-lock

c035ed2

This was referenced Aug 1, 2025

header/header eslint rule can blow away top-comments #2967

Open

feat(semantic-conventions): add 'EVENT_*' exports for event names open-telemetry/opentelemetry-js#5832

Merged

trentm reviewed Aug 1, 2025

View reviewed changes

anuraaga added 2 commits August 7, 2025 11:48

Owner

d6c427b

Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…

36dd80b

…y-js-contrib into openai-inst

github-actions bot assigned trentm Aug 7, 2025

anuraaga marked this pull request as ready for review August 7, 2025 03:09

trentm approved these changes Aug 19, 2025

View reviewed changes

.github/component_owners.yml Show resolved Hide resolved

anuraaga and others added 2 commits August 20, 2025 10:41

Update .github/component_owners.yml

e89981e

Co-authored-by: Trent Mick <[email protected]>

Merge branch 'main' into openai-inst

1cd03cb

github-actions bot assigned seemk Aug 20, 2025

legendecas approved these changes Aug 20, 2025

View reviewed changes

pichlermarc enabled auto-merge (squash) August 21, 2025 13:00

Merge branch 'main' into openai-inst

6d77251

pichlermarc merged commit 3e575ba into open-telemetry:main Aug 21, 2025
21 checks passed

dyladan mentioned this pull request Aug 21, 2025

chore: release main #2982

Open

feat(instrumentation-openai): add instrumentation of openai SDK #2941

feat(instrumentation-openai): add instrumentation of openai SDK #2941

Conversation

anuraaga commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which problem is this PR solving?

Short description of the changes

Uh oh!

anuraaga Jul 18, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

trentm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

seemk Jul 26, 2025

Choose a reason for hiding this comment

Uh oh!

trentm Jul 30, 2025

Choose a reason for hiding this comment

Uh oh!

anuraaga Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

trentm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

trentm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

trentm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

trentm commented Aug 1, 2025

Uh oh!

trentm Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anuraaga commented Aug 7, 2025

Uh oh!

trentm commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seemk commented Aug 19, 2025

Uh oh!

trentm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

anuraaga commented Aug 20, 2025

Uh oh!

Uh oh!

Uh oh!

anuraaga commented Jul 17, 2025 •

edited

Loading

codecov bot commented Jul 18, 2025 •

edited

Loading

trentm Aug 1, 2025 •

edited

Loading

trentm commented Aug 19, 2025 •

edited

Loading