High-Traffic Direct Ingestion

[Vail-X]

"Hi Jaeger Team,

I am deploying Jaeger in a Kubernetes cluster with a large number of microservices/applications that want to send traces. We intend to have all applications send traces directly to the central jaeger-collector Service (Deployment).

My primary concern is high load:

• Given many (lets say tens or hundreds) of applications pointing to the Collector Service, would a default Collector setup be immediately overwhelmed?
• If we anticipate high load, should we focus on horizontally scaling the Jaeger Collector Deployment (replicaCount > 1), or are there specific configuration tunings that would help?
• Since the jaeger-agent is deprecated, is there a recommended alternative design pattern to tackle this problem?

[dosubot[bot]]

Hi @Vail-X! I'm Dosu and I’m helping the jaeger team.

A default Jaeger Collector setup is likely to be overwhelmed if you have tens or hundreds of applications sending traces directly, because the default queue size (2000) and worker count (50) are tuned for modest workloads. Under high load, the collector can drop spans when its internal queue fills up, and you may see errors or increased latency. You can tune these parameters—queue size, number of workers, and enable dynamic queue sizing—to increase ingestion capacity, but you’ll also need to provision enough CPU and memory resources for the collector pods [source].

Horizontal scaling (increasing replicaCount) is recommended, since the collector is stateless and can be scaled out to distribute load. In Kubernetes, use a Service to load-balance traffic across collector pods. For very high throughput, you may need to tune queue and batch settings, and monitor metrics for dropped spans and queue health [source].

Since jaeger-agent is deprecated, the recommended pattern is to use the OpenTelemetry Collector as a buffering and batching layer between your applications and Jaeger Collector. This intermediary can handle batching, retries, and tail-based sampling, and helps prevent overload on the Jaeger backend. The modern architecture is: apps → OpenTelemetry Collector (OTLP) → Jaeger Collector → storage [source], [source].

Monitor metrics like otelcol_receiver_accepted_spans, otelcol_exporter_send_failed_spans, and storage latency to catch bottlenecks early. If you see “sending queue is full” errors, increase the queue size and number of consumers in your collector/exporter config [source].

Real-world deployments have handled hundreds of services and hundreds of GBs of trace data per day, but only with careful tuning and scaling [source]. Adaptive sampling is not fully supported with OpenTelemetry SDKs yet, so sampling strategies may be limited in high-load scenarios.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[Vail-X]

Is it recoomended to run OpenTelemetry Collector as daemonset or deployment?

[yurishkuro]

is it more useful as a host agent, because it allows you to apply various attribute enrichments known on the host but unknown in a separately deployed cluster ("service"). If you run it as a service then there's not much value it would provide over just running Jaeger collector as a service.

[Vail-X]

I see thank you for this information, it was really helpful!

[Vail-X]

Would deploying Jaeger Collector as a Daemonset or Deployment with high horizontal scalability work in this case? this would put strain in the storage though right meaning might need to do smth about that

[yurishkuro]

Jaeger is meant to be deployed as horizontally scaled, but you have to size the storage accordingly

[Vail-X]

Gott itt thank youu