Multi-Project Routing with OpenTelemetry Configuration

[KeerthikaNachimuthu]

Describe your question

Question

I'm trying to map multiple Langfuse projects to different subprojects within my application using OpenTelemetry configuration.

Current Setup

I have the following OTLP exporter configuration:

otel:
  exporters:
    - otlp:
        endpoint: "https://cloud.langfuse.com/api/public/otel/v1/traces"
        headers:
          Authorization: "Basic <encoded-keys>"
  enabled: true
  service_name: "MyApp"

Goal
I need to route traces from different subprojects/tenants within my application to different Langfuse projects. Each subproject should send traces to its own dedicated Langfuse project.

Questions
What's the recommended approach for multi-project routing using OpenTelemetry configuration?
Should I use multiple OTLP exporters with different credentials, or is an OpenTelemetry Collector with filtering the better approach?
Are there any documented examples or best practices for this use case?
Additional Context
I'm using pure OpenTelemetry (not the Langfuse Python SDK) and would prefer a configuration-based solution if possible.

Langfuse Cloud or Self-Hosted?

Langfuse Cloud

If Self-Hosted

No response

If Langfuse Cloud

No response

SDK and integration versions

No response

Pre-Submission Checklist

• I have checked for existing issues/discussions and consulted Langfuse AI.

[inkeep-ai-assistant[bot]]

Multi-Project Routing with OpenTelemetry

For routing traces to multiple Langfuse projects using OpenTelemetry configuration, there are two primary approaches:

1. Multiple OTLP Exporters (Recommended for Configuration-Based Setup)

You can configure multiple OTLP exporters, each with different credentials pointing to different Langfuse projects(1). Each exporter uses Basic Auth with project-specific API keys.

The authentication header format is:

Authorization: Basic {base64(public_key:secret_key)}

(1)

For your YAML configuration, this would look like:

otel:
  exporters:
    - otlp:
        endpoint: "https://cloud.langfuse.com/api/public/otel/v1/traces"
        headers:
          Authorization: "Basic <project-a-encoded-keys>"
    - otlp:
        endpoint: "https://cloud.langfuse.com/api/public/otel/v1/traces"
        headers:
          Authorization: "Basic <project-b-encoded-keys>"

Important: You'll need to implement logic to route specific traces to specific exporters based on your subproject/tenant context(1).

2. OpenTelemetry Collector with Filtering

You can use an OpenTelemetry Collector with the filterprocessor to selectively send spans to different Langfuse projects(1). This approach allows you to filter spans based on attributes, span names, and other criteria.

Example collector configuration:

processors:
  filter/openaisystem:
    error_mode: ignore
    traces:
      span:
        - 'attributes["gen_ai.system"] != "openai"'

exporters:
  otlphttp/langfuse:
    endpoint: "https://cloud.langfuse.com/api/public/otel"
    headers:
      Authorization: "Basic ${AUTH_STRING}"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [filter/openaisystem]
      exporters: [otlphttp/langfuse]

(1)

Caution: Filtering at the span level may risk incomplete traces, so be careful with complex filter rules(1). Langfuse requires that a root span is sent to ensure traces are created correctly(1).

Best Practices

• Use project-scoped API keys (not organization keys) for each Langfuse project(1), (2)
• Ensure the endpoint is https://cloud.langfuse.com/api/public/otel or https://cloud.langfuse.com/api/public/otel/v1/traces for Langfuse Cloud(1)
• The Authorization header must use Basic Auth with base64-encoded credentials in the format public_key:secret_key(1), (2)

For multi-tenant scenarios, you can also set up separate projects within the same Langfuse instance and route different tenants to these projects(3).

📚 Sources:

1. LLM Observability via OpenTelemetry
2. bug: self-hosted option does not work with tracing out of the box
3. Multi-Tenancy and others

---

Have another question? Just tag @inkeep.

[hassiebp]

Please see our docs on multi project setups and please let us know your open questions: https://langfuse.com/docs/observability/sdk/advanced-features#multi-project-setup-experimental