Feature Request: Forward custom headers to upstream URLs in Gateway

[13Mai13]

Is your feature request related to a problem?

Yes. When using any-llm-gateway with a custom api_base URL pointing to an intermediary service (e.g., an authentication proxy), the gateway does not forward custom headers from incoming requests to the upstream URL. This prevents intermediary services from receiving necessary authentication or metadata headers.

Current Behavior

• ✅ Gateway successfully forwards requests to custom URLs
• ✅ Can use dummy API keys for providers (intermediary handles real credentials)
• ❌ Does NOT forward any extra headers from incoming requests

Looking at src/any_llm/gateway/routes/chat.py (lines 194-196), the gateway only passes credentials from config:

completion_kwargs = request.model_dump()
completion_kwargs.update(credentials)

Headers from the incoming HTTP request are stripped/not forwarded.

Impact

Intermediary services cannot authenticate or process requests because required headers from the original request are missing.

Example flow:

Client → [Headers: Authorization, X-Custom-Auth] → any-llm-gateway → [Headers: ???] → Intermediary Service
                                                                        ↑
                                                        Headers not forwarded, service can't process!

Describe the solution you'd like

Allow configuring specific headers to forward from incoming requests to upstream URLs.

Suggested configuration (config.yml):

providers:
  openai:
    api_key: "dummy"  # Intermediary handles real credentials
    api_base: "https://my-intermediary-service.com/v1"
    forward_headers:
      - "Authorization"
      - "X-Custom-Auth"
      - "X-Request-ID"
      - "X-Trace-ID"

Alternative: Global forwarding

# Forward these headers to all providers
forward_headers:
  - "X-Request-ID"
  - "X-Trace-*"  # wildcard pattern

providers:
  openai:
    api_key: "dummy"
    api_base: "https://my-intermediary-service.com/v1"
    # Per-provider overrides/additions
    forward_headers:
      - "Authorization"

Implementation approach:

1. Add forward_headers config option to GatewayConfig (global and/or per-provider)
2. Extract matching headers from incoming Request object in chat.py
3. Pass them to the provider via default_headers in client initialization

Describe alternatives you've considered

• Reverse proxy (nginx/Traefik) in front of gateway - but this doesn't allow selective forwarding based on gateway config and adds unnecessary complexity
• Modifying provider credentials to include headers - but this is static and doesn't support request-specific values like trace IDs

Use Cases

• Authentication proxies that handle credential management/rotation
• Distributed tracing (X-Request-ID, X-Trace-ID, X-Correlation-ID)
• Multi-tenant setups requiring request attribution headers
• Enterprise deployments with intermediary services for logging, monitoring, or access control
• Custom provider deployments behind proxies that require specific headers
• Observability platforms (Datadog, OpenTelemetry, etc.)

Additional context

This is particularly useful for architectures where:

• An intermediary service sits between the gateway and the actual LLM provider
• The intermediary handles authentication/credential injection
• The gateway uses dummy credentials and relies on forwarded headers for the intermediary to process requests correctly

[njbrake]

HI @13Mai13 thank you for the feature request. By passing additional headers from the client through to the LLM provider, it seems to me that then we would be adding extra dependencies between the client that would tie them to a specific provider. The idea of any-llm-gateway is to be the proxy such that the client can easily toggle between providers without the client needing to change different settings. Is there a way to accomplish what you need by setting something in the any-llm-gateway such that the client doesn't need to pass any extra headers?

If you can provide some more specific details about your use case, we can help determine where the custom headers would make sense to add.

[iamgautamraj]

Hi @njbrake , your point about keeping the client provider-agnostic makes total sense.

What if instead of forwarding client headers, the gateway itself injects headers from config? The client wouldn't need to pass anything extra - headers would be configured at the gateway level:

providers:
  openai:
    api_key: "dummy"
    api_base: "https://auth-proxy.internal/v1"
    inject_headers:
      Authorization: "${INTERNAL_AUTH_TOKEN}"
      X-Trace-ID: "auto"

This way:

• Client stays untied (just sends model + messages)
• Gateway admin controls what gets injected per-provider
• Switching providers = just change gateway config, no client changes

This fits the existing providers config pattern and would use the default_headers approach already in place for GatewayProvider.

Happy to put together a PR for this if you think it's a reasonable direction!

[njbrake]

Thanks @iamgautamraj: any-llm should already allow this, we let you pass custom client args in when you create an AnyLLM client. I'll add a unit test which should verify the feature, and we should also add some docs to explain how this works. Thanks! 🚀