Jaeger V2: Is there a configuration parameter to control retries when connecting to Elasticsearch? #8057
Replies: 1 comment
-
|
Hi @RishithReddyRR! I'm Dosu and I’m helping the jaeger team. No, Jaeger V2 does not currently have a configuration parameter for connection retries to Elasticsearch. The retry mechanism is explicitly disabled in the code because export operations may not be idempotent across all storage backends. Workaround: You can try setting jaeger_storage:
backends:
collector_storage:
elasticsearch:
server_urls:
- https://elastic-search:9200
disable_health_check: trueThis disables the health check that's causing the timeout error, though some cluster-level operations may still attempt to connect. Alternative approach: For production resilience, consider using Kafka as an intermediary buffer between collectors and Elasticsearch storage—this provides durability when Elasticsearch is temporarily unavailable. To reply, just mention @dosu. How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
I am using Elasticsearch as the storage backend for Jaeger in Kubernetes.In my Jaeger pod I have an init container that checks Elasticsearch availability. During a Helm upgrade, sometimes the init container succeeds, but shortly after that Elasticsearch becomes temporarily unavailable before jaeger binary gets started in main container.
When this happens, Jaeger logs the following error and exits:
failed to start extensions: failed to initialize storage 'collector_storage': failed to create Elasticsearch client: health check timeout: Head \"https://elastic-search:9200\": dial tcp 10.104.37.149:9200: connect: connection refused: no Elasticsearch node availableBecause Jaeger exits, the Kubernetes probes fail and the container keeps restarting.
I want to know if there is any parameter I can configure in ConfigMap so that Jaeger retries the Elasticsearch connection instead of exiting immediately when Elasticsearch is temporarily down.
Is retry configuration supported in Jaeger V2?
Beta Was this translation helpful? Give feedback.
All reactions