Skip to content

sync span based baggage APIs with global baggage APIs #7453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ida613 wants to merge 10 commits into
base: master
Choose a base branch
from
+51 −97
Open

sync span based baggage APIs with global baggage APIs #7453

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
f6e65d6
sync span based baggage apis with global baggage apis
ida613 Feb 6, 2026
5b5901b
fix linter errors
ida613 Feb 6, 2026
0251bf5
PR comment round 1
ida613 Feb 9, 2026
0767df5
fix shallow copy of baggage
ida613 Feb 9, 2026
704540a
swap structuredClone with rfdc
ida613 Feb 9, 2026
4522c66
let dd baggage always override otel baggage
ida613 Feb 10, 2026
0bda3cf
let whichever baggage is set latest take precedence
ida613 Feb 10, 2026
6af793c
use global api (draft)
ida613 Feb 13, 2026
5e35934
update unit test
ida613 Feb 17, 2026
2fa1be7
fix linter errors
ida613 Feb 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 17 additions & 3 deletions packages/dd-trace/src/opentelemetry/context_manager.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

const { trace, ROOT_CONTEXT, propagation } = require('@opentelemetry/api')
const { storage } = require('../../../datadog-core')
const { getAllBaggageItems, setBaggageItem, removeAllBaggageItems } = require('../baggage')

const tracer = require('../../')
const SpanContext = require('./span_context')
Expand All @@ -11,6 +12,15 @@ class ContextManager {
this._store = storage('opentelemetry')
}

_mergeGlobalBaggageWith (baggages) {
baggages = baggages || {}
const globalActiveBaggages = getAllBaggageItems()
for (const [key, value] of Object.entries(globalActiveBaggages)) {
if (!baggages[key]) baggages[key] = value
}
return baggages
}

// converts dd to otel
active () {
const store = this._store.getStore()
Expand All @@ -21,7 +31,7 @@ class ContextManager {

// If stored span wraps the active DD span, prefer the stored context
if (storedSpan && storedSpan._ddSpan === activeSpan) {
const baggages = JSON.parse(activeSpan.getAllBaggageItems())
const baggages = this._mergeGlobalBaggageWith(JSON.parse(activeSpan.getAllBaggageItems()))
if (Object.keys(baggages).length > 0) {
const entries = {}
for (const [key, value] of Object.entries(baggages)) {
Expand All @@ -34,7 +44,7 @@ class ContextManager {
}

if (!activeSpan) {
const storedBaggageItems = storedSpan?._spanContext?._ddContext?._baggageItems
const storedBaggageItems = this._mergeGlobalBaggageWith(storedSpan?._spanContext?._ddContext?._baggageItems)
if (storedBaggageItems) {
const baggages = storedBaggageItems
const entries = {}
Expand All @@ -54,7 +64,7 @@ class ContextManager {
}

// Convert DD baggage to OTel format
const baggages = JSON.parse(activeSpan.getAllBaggageItems())
const baggages = this._mergeGlobalBaggageWith(JSON.parse(activeSpan.getAllBaggageItems()))
const hasBaggage = Object.keys(baggages).length > 0
let otelBaggages
if (hasBaggage) {
Expand Down Expand Up @@ -86,6 +96,10 @@ class ContextManager {
if (baggages) {
baggageItems = baggages.getAllEntries()
}
removeAllBaggageItems()
for (const baggage of baggageItems) {
setBaggageItem(baggage[0], baggage[1].value)
}
Copy link
Member

@BridgeAR BridgeAR Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate about this? I am unsure what this does.

Copy link
Collaborator Author

@ida613 ida613 Feb 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is calling the global baggage APIs to make sure the currently active baggage is in sync with baggage on the context. It essentially does the same thing as these operations but for global baggage

Copy link
Member

@BridgeAR BridgeAR Feb 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should Otel baggage overrule DD baggage? Should it be merged? Do we have any documentation or RFCs around that or was there any discussion around that?

Copy link
Collaborator Author

@ida613 ida613 Feb 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. The code lets whichever comes later override, but I'm not sure if this is what's intended. @rachelyangdog @zacharycmontoya do you think we need to discuss this before proceeding?

Copy link
Collaborator Author

@ida613 ida613 Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so turns out dd baggage should always override otel baggage, updated the code

Copy link
Collaborator Author

@ida613 ida613 Feb 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I support this approach. Legacy baggage won't be included in this support. We will only keep the Current OTel API Baggage in sync with the global Datadog baggage that's tracked independently of spans.

Thank you Zach! I can do this no problem, but currently in node, dd and otel baggages only get synced when we call active or with. In order to do what dotnet does (syncing on each individual operation), we would have to replace the otel baggage operations within dd-trace, which will involve a ton of work. Is it acceptable to only sync when calling with or active?

Did we already establish a precedence for this scenario or are we only now adding that? My preference would be to prefer global baggage, aka from the baggage HTTP header. How feasible would that be and do you see any issues with that preference?

Sounds good with me, it's not hard to do. I'll work on it :)

Copy link
Contributor

@zacharycmontoya zacharycmontoya Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think it should be fine to only update when calling active or with. Let me describe the interactions I expect, but if my understanding is wrong please point it out so we're operating off the same understanding.

DD API

Each time the user wants to read/write to the active global baggage, they call one of the Baggage API's here. Notably, when clearing or setting baggage, these API's will automatically update the current baggage before returning to the caller. As long as we instrument each time active or with is called in the OpenTelemetry context manager, we don't need to update any code here.

OTel API

Each time the user wants to read/write to the active global baggage, they will need to get the current baggage by calling getActiveBagage which gets the current context by calling active. Then, once the user has updated the baggage and wants to set it as current, the user must attach the baggage to a Context using the OTel API setBaggage and call the api.context.with API with the Context and a callback, so the baggage will become active.

So with this flow, we just need our active operation to copy over the Datadog global baggage to overwrite all the OpenTelemetry baggage in the context and our with operation to copy over the OpenTelemetry baggage to override all the Datadog baggage.

And actually, this is exactly what my .NET implementation does! 😄 It just turns out that the .NET OTel Baggage API's directly set the "current" OTel baggage API within each API call (Set/Remove) so we just have to do the active/with implementation in more places

Copy link
Collaborator Author

@ida613 ida613 Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks Zach! that's my understanding too. I've updated the code accordingly

Copy link
Collaborator Author

@ida613 ida613 Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zacharycmontoya turns out legacy baggage and the new baggage are extracted/injected and propagated independently, so there will be never be a key conflict. But the legacy baggage never interact with the otel drop in code, is this okay?

Copy link
Contributor

@zacharycmontoya zacharycmontoya Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's okay 👍🏼

if (span && span._ddSpan) {
// does otel always override datadog?
span._ddSpan.removeAllBaggageItems()
Expand Down
4 changes: 4 additions & 0 deletions packages/dd-trace/src/opentracing/span.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ const log = require('../log')
const { storage } = require('../../../datadog-core')
const telemetryMetrics = require('../telemetry/metrics')
const { getValueFromEnvSources } = require('../config/helper')
const { setBaggageItem, removeBaggageItem, removeAllBaggageItems } = require('../baggage')
const SpanContext = require('./span_context')

const tracerMetrics = telemetryMetrics.manager.namespace('tracers')
Expand Down Expand Up @@ -163,6 +164,7 @@ class DatadogSpan {

setBaggageItem (key, value) {
this._spanContext._baggageItems[key] = value
setBaggageItem(key, value)
return this
}

Expand All @@ -176,10 +178,12 @@ class DatadogSpan {

removeBaggageItem (key) {
delete this._spanContext._baggageItems[key]
removeBaggageItem(key)
}

removeAllBaggageItems () {
this._spanContext._baggageItems = {}
removeAllBaggageItems()
}

setTag (key, value) {
Expand Down
10 changes: 7 additions & 3 deletions packages/dd-trace/test/opentelemetry/context_manager.spec.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,9 @@ describe('OTel Context Manager', () => {
assert.deepStrictEqual(JSON.parse(ddSpan.getAllBaggageItems()), { key1: 'dd1' })
api.context.with(contextWithUpdatedBaggages, () => {
assert.deepStrictEqual(JSON.parse(ddSpan.getAllBaggageItems()), { key1: 'otel1', key2: 'otel2' })
ddSpan.setBaggageItem('key2', 'dd2')
})
ddSpan.setBaggageItem('key2', 'dd2')
tracer.scope().activate(ddSpan, () => {
assert.deepStrictEqual(propagation.getActiveBaggage().getAllEntries(),
[['key1', { value: 'otel1' }], ['key2', { value: 'dd2' }]]
)
Expand All @@ -192,8 +194,10 @@ describe('OTel Context Manager', () => {
assert.deepStrictEqual(JSON.parse(ddSpan.getAllBaggageItems()), { key1: 'dd1', key2: 'dd2' })
api.context.with(contextWithUpdatedBaggages, () => {
assert.deepStrictEqual(JSON.parse(ddSpan.getAllBaggageItems()), { key2: 'dd2' })
ddSpan.removeBaggageItem('key2')
assert.deepStrictEqual(propagation.getActiveBaggage().getAllEntries(), [])
})
ddSpan.removeBaggageItem('key2')
tracer.scope().activate(ddSpan, () => {
assert.deepStrictEqual(propagation.getActiveBaggage(), undefined)
})
})

Expand Down
Loading