Capturing HTTP Headers as span attributes not working

[azunna1]

Describe the bug

I am using autoinstrumentation for multiple springboot applications and trying to capture some request headers as span attributes following the guide here but it does not work.

I am not exactly sure what the problem is.

Steps to reproduce

I am using the following:

Deployment Environment - Kubernetes
Instrumentation Method - AutoInstrumentation using the Opentelemetry Operator
Java Agent Version - 2.18.1
instrumentation.yaml

apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: default-instrumentation
spec:
  propagators:
    - tracecontext
    - baggage
  sampler:
    type: parentbased_traceidratio
    argument: "1"
  env:
    - name: OTEL_EXPORTER_OTLP_TIMEOUT
      value: "600000"
    - name: OTEL_LOG_LEVEL
      value: fatal
    - name: OTEL_METRICS_EXPORTER
      value: prometheus
    - name: OTEL_EXPORTER_PROMETHEUS_HOST
      value: "0.0.0.0"
    - name: OTEL_EXPORTER_PROMETHEUS_PORT
      value: "9464"
    - name: OTEL_LOGS_EXPORTER
      value: none
    - name: OTEL_EXPERIMENTAL_EXPORTER_OTLP_RETRY_ENABLED
      value: "true"
    - name: OTEL_EXPORTER_OTLP_PROTOCOL
      value: grpc
    - name: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT
      value: http://traces-collector-headless.opentelemetry.svc:4317
  java:
    image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-java:2.18.1
    env:
      - name: OTEL_INSTRUMENTATION_SPRING_SCHEDULING_ENABLED
        value: "false"
      - name: OTEL_JAVAAGENT_LOGGING
        value: "none"
      - name: OTEL_INSTRUMENTATION_HTTP_SERVER_CAPTURE_REQUEST_HEADERS
        value: "x-user-id,x-org-id"
      - name: OTEL_INSTRUMENTATION_HTTP_CLIENT_CAPTURE_REQUEST_HEADERS
        value: "x-user-id,x-org-id"

Expected behavior

The headers should show up as span attributes.

Actual behavior

Http headers missing from span attributes

Span #1
    Trace ID       : 60fe825ddc5593c3cd451fb7bdb37ac8
    Parent ID      : e3cdb8736ade95c8
    ID             : 0e9b785764c6f383
    Name           : GET /api/v1/path
    Kind           : Server
    Start time     : 2025-08-07 13:21:34.832875481 +0000 UTC
    End time       : 2025-08-07 13:21:35.339639781 +0000 UTC
    Status code    : Unset
    Status message :
Attributes:
     -> http.request.method: Str(GET)
     -> http.route: Str(/api/v1/path)
     -> server.address: Str(my-service.namespace.svc)
     -> client.address: Str(x.x.x.x)
     -> network.peer.address: Str(127.0.0.6)
     -> url.path: Str(/api/v1/path)
     -> server.port: Int(8080)
     -> thread.id: Int(60)
     -> http.response.status_code: Int(200)
     -> network.protocol.version: Str(1.1)
     -> user_agent.original: Str(PostmanRuntime/7.44.1)
     -> url.scheme: Str(https)
     -> thread.name: Str(http-nio-8080-exec-9)
     -> network.peer.port: Int(39137)

Javaagent or library instrumentation version

v2.18.1

Environment

JDK: Eclipse 17
OS: Ubuntu 20.04 Spring Boot 3.5.4

Additional context

No response

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[laurit]

My first guess would be that this is a misconfiguration. If you believe you have found a bug then please provide a minimal sample application that reproduces the issue. Don't use kubernetes.

[azunna1]

Closing this because i got it working now.

[Glaha]

Hi, could you share how to fix this issue? I also met this problem. Thank you.@azunna1

Closing this because i got it working now.

[azunna1]

Hi, could you share how to fix this issue? I also met this problem. Thank you.@azunna1

Closing this because i got it working now.

I honestly can't recall what the fix was. My guess is that it started working on it's own or the otel collector was dropping those spans. I made some changes to the collector as well so it might have been that. This is the current instrumentation config i'm using

java:
    image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-java:2.18.1
    env:
      - name: OTEL_INSTRUMENTATION_SPRING_SCHEDULING_ENABLED
        value: "false"
      - name: OTEL_INSTRUMENTATION_MICROMETER_ENABLED
        value: "true"
      - name: OTEL_JAVAAGENT_LOGGING
        value: "none"
      - name: OTEL_INSTRUMENTATION_HTTP_SERVER_CAPTURE_REQUEST_HEADERS
        value: "x-user-id,x-org-id"
      - name: OTEL_INSTRUMENTATION_HTTP_CLIENT_CAPTURE_REQUEST_HEADERS
        value: "x-user-id,x-org-id"

You can also check that the collector isn't dropping those spans.

[Glaha]

Thank you very much.
My collector config dropped my spans. That's my problem.

Hi, could you share how to fix this issue? I also met this problem. Thank you.@azunna1

Closing this because i got it working now.

I honestly can't recall what the fix was. My guess is that it started working on it's own or the otel collector was dropping those spans. I made some changes to the collector as well so it might have been that. This is the current instrumentation config i'm using

java:
    image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-java:2.18.1
    env:
      - name: OTEL_INSTRUMENTATION_SPRING_SCHEDULING_ENABLED
        value: "false"
      - name: OTEL_INSTRUMENTATION_MICROMETER_ENABLED
        value: "true"
      - name: OTEL_JAVAAGENT_LOGGING
        value: "none"
      - name: OTEL_INSTRUMENTATION_HTTP_SERVER_CAPTURE_REQUEST_HEADERS
        value: "x-user-id,x-org-id"
      - name: OTEL_INSTRUMENTATION_HTTP_CLIENT_CAPTURE_REQUEST_HEADERS
        value: "x-user-id,x-org-id"

You can also check that the collector isn't dropping those spans.