-
Notifications
You must be signed in to change notification settings - Fork 492
Description
Component(s)
No response
Request
Add forward_to parameter to stage.metrics block in loki.process, allowing metrics to be pushed directly to prometheus.remote_write receivers instead of (or in addition to) /metrics endpoint exposure
Use case
Currently, stage.metrics in loki.process only exposes metrics via /metrics endpoint for scraping. This creates operational challenges when dealing with high-cardinality metrics derived from logs:
❌ Requires external scraping (VMAgent or self-scraping via prometheus.scrape)
❌ Cannot isolate metric streams - all metrics mixed on same endpoint
❌ Adds unnecessary latency - scrape_interval delays (15-30s typically)
❌ Architectural inconsistency - logs are pushed (loki.write) but metrics must be pulled
❌ Resource inefficiency - requires deployment and maintenance of separate scraping infrastructure
We operate a multi-tenant Kubernetes platform processing logs from 1000+ pods per cluster. Using stage.metrics, we generate ~500K metric series from structured logs.
Current architecture requires:
VMAgent to scrape Alloy's /metrics endpoint
All tenant metrics mixed with Alloy's operational metrics
Cannot route tenant A metrics to cluster X and tenant B to cluster Y
Tip
React with 👍 if this issue is important to you.