Skip to content

Conversation

srprash
Copy link
Contributor

@srprash srprash commented Jul 9, 2025

Issue

The OTEL_PROPAGATORS configured in the ADOT do not actually take effect. The configured propagators are still tracecontext and baggage which are the default ones configured by OTel SDK. This causes ADOT to ignore the x-amzn-trace-id trace header causing broken traces if there is only XRay format trace being propagated.

Root Cause

  • The opentelemetry.propagate module has the logic to load the propagators configured via the OTEL_PROPAGATORS env variable. If the env variable is not set, then it sets up the default propagators.
  • This module is loaded and run very early in the OTel startup logic even before the ADOT distro code is run which sets the OTEL_PROPAGATORS env variable. This point is too late to modify the OTEL_PROPAGATORS env variable.

Solution

Set the OTEL_PROPAGATORS env variable in ADOT distro and reload the opentelemetry.propagate module (using importlib.reload) which will force it to reinitialize the intended propagators.

Related Note

With this fix, the ADOT configured propagators will take effect, which currently does not include baggage and will break some existing functionality that depends on baggage. So I'm also also updating the propagators set by the ADOT to:

  • include baggage
  • drop the B3 propagators as ADOT JS did
  • reorder to baggage, xray, tracecontext. This will be the order we follow for other ADOT SDKs too (JS already in progress).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@srprash srprash force-pushed the fix_propagator branch 3 times, most recently from 9f125fc to d8ce58a Compare July 9, 2025 20:29
@srprash srprash changed the title Reload propagators for distro config to take effect Reload propagators for distro config + Reorder propagators Jul 9, 2025
@srprash srprash marked this pull request as ready for review July 9, 2025 21:44
@srprash srprash requested a review from a team as a code owner July 9, 2025 21:44
@srprash srprash merged commit 1324488 into main Jul 9, 2025
13 checks passed
@srprash srprash deleted the fix_propagator branch July 9, 2025 23:34
jj22ee added a commit that referenced this pull request Jul 11, 2025
Related issue in ADOT:
#421
Opened issue in OTel Python:
open-telemetry/opentelemetry-python#4679


By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.

Co-authored-by: Jonathan Lee <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants