feat(llmobs): implement multi-tenant routing context support

[PROFeNoM]

What does this PR do?

Adds multi-tenant API key support for LLM Observability, enabling platforms to route LLMObs telemetry to different Datadog organizations from a single process.

New API:

await llmobs.withRoutingContext({ ddApiKey, ddSite }, async () => {
  // All LLMObs spans here route to the specified Datadog org
})

Motivation

Multi-tenant AI platforms need to send LLM Observability data to their customers' Datadog organizations. This enables:

• Customer-specific trace visibility
• Data isolation between tenants
• Compliance with data residency requirements (different DD sites per customer)

Implementation

  withRoutingContext({ ddApiKey, ddSite }, fn)
      ↓ AsyncLocalStorage
  registerLLMObsSpan()
      ↓ captures routing to span tags
  span.finish()
      ↓
  SpanProcessor.process()
      ↓ extracts routing from tags
  writer.append(event, routing)
      ↓ routes to correct buffer
  setInterval flush()
      ↓ flushes each buffer to its endpoint

Key Changes:

• withRoutingContext(options, fn) method added to LLMObs SDK (uses AsyncLocalStorage)
• Routing context captured at span creation time
• BaseLLMObsWriter refactored to multi-buffer architecture (one buffer per API key)
• Each buffer flushes independently to its corresponding endpoint
• Agentless mode only

Testing

• writers/multi-tenant.spec.js - Tests for buffer routing, endpoint construction, API key isolation, context nesting, and concurrency

[codecov]

Codecov Report

❌ Patch coverage is 51.28205% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.12%. Comparing base (b9e9246) to head (a3d89ff).

Files with missing lines	Patch %	Lines
packages/dd-trace/src/llmobs/writers/base.js	59.25%	22 Missing ⚠️
packages/dd-trace/src/llmobs/sdk.js	0.00%	11 Missing ⚠️
packages/dd-trace/src/llmobs/tagger.js	40.00%	3 Missing ⚠️
packages/dd-trace/src/llmobs/index.js	0.00%	1 Missing ⚠️
packages/dd-trace/src/llmobs/noop.js	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7158      +/-   ##
==========================================
- Coverage   86.25%   86.12%   -0.14%     
==========================================
  Files         513      513              
  Lines       22054    22106      +52     
==========================================
+ Hits        19023    19039      +16     
- Misses       3031     3067      +36     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[PROFeNoM]

@codex review

[github-actions]

Overall package size

Self size: 4.41 MB
Deduped: 5.24 MB
No deduping: 5.24 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 2.0.0 | 68.46 kB | 797.03 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

[chatgpt-codex-connector]

To use Codex here, create a Codex account and connect to github.

[datadog-datadog-prod-us1]

⚠️ Tests

✨ Fix all issues with Cursor

⚠️ Warnings

❄️ 1 New flaky test detected

[email protected] esm early flake detection does not run EFD if the known tests request fails from integration-tests/cypress/cypress.spec.js (Datadog) (Fix with Cursor)

Cannot read properties of undefined (reading 'content')

ℹ️ Info

🧪 All tests passed

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a3d89ff | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

[pr-commenter]

Benchmarks

Benchmark execution time: 2026-01-20 15:52:12

Comparing candidate commit a3d89ff in PR branch alex/MLOB-4999_multi-tenant-routing-context-support with baseline commit b9e9246 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 233 metrics, 27 unstable metrics.

[BridgeAR]

Please use private properties instead of underscores for all changed properties and new ones. Changing old ones is nice to have as separate commit :)

Please also add a comment to our Claude.md file that highlights this. Otherwise new code changes will likely come up by the AI that use the underscore instead.

[PROFeNoM]

Done. Converted to # private syntax where possible (#buffers, #getMaskedRoutingKey, #cleanupEmptyBuffers, #getUrlForRouting)

Kept _getRoutingKey and _getOrCreateBuffer with underscore because LLMObsSpanWriter (subclass) needs to access them.

Also added the convention to AGENTS.md (symlink to claude.md ?) under "Class Properties and Methods".

[PROFeNoM]

FYI, I did keep things like get _buffer, set _buffer with the underscore because they are backward compatibility shims. You were not thinking about changing them as well, right?

[sabrenner]

mostly small comments but one larger implementation comment i'm curious on your thoughts on. but overall the logic makes sense to me and the sdk api change looks good pending some type definition updates!

[sabrenner]

left a question on the test file added!

[sabrenner]

looks good to me now! just nits, plus some of the nested routing context tests are failing, i'm happy to help with looking into those, so lmk

[sabrenner]

this looks great now! just one small comment (barring no changes for supporting evals in this as well, see below, i promise it's the last one 😭) about the function name we should consider before shipping, but then i'll approve.

i followed up offline, but if we're not supporting evals in this pr we should call that out for those looking over release notes

[sabrenner]

really awesome job!! 🚢