-
Notifications
You must be signed in to change notification settings - Fork 543
Description
Component(s)
prometheus.receive_http
What's wrong?
Problem:
Metrics received by prometheus.receive_http (ex: prometheus_receive_http_receiver_classic_metrics_received_samples_total) are NOT exposed on Alloy's standard /metrics endpoint, blocking external monitoring.
Impact:
Impossible monitoring of proxy/gateway metrics without local TSDB (Mimir/Prometheus).
v0 API endpoints are internal debug only, not scrapable by external Prometheus.
Use Case:
Get recieved metrics from http through the UI
Or, get recieved metrics through a prometheus.echo that output in the /metrics or /api/v0/component/prometheus.echo./metrics
Or get metrics from a scraper
Steps to reproduce
use Alloy as a proxy.
Create a http receiver and send metrics from alloy agent to alloy proxy gateway
❌/api/v0/component/prometheus.receive_http.receiver_classic_metrics/metrics → 404
❌ /api/v0/component/prometheus.echo.received_metrics_exposure/metrics → 404
❌ /api/v0/component/prometheus.scrape.echo_scraper/metrics → 404
❌ no received metrics in the /metrics of alloy UI
System information
Linux 3.10.0-1160.145.1.el7.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
Software version
Grafana Alloy 1.13.2
Configuration
// activer le mode debug pour afficher toutes les métriques passantes par le proxy
livedebugging {
enabled = true
}
// reception des métriques de traces
otelcol.receiver.otlp "receiver_traces_metrics" {
grpc {
endpoint = "0.0.0.0:4319"
}
http {
endpoint = "0.0.0.0:4320"
}
output {
metrics = [otelcol.exporter.otlphttp.sender_to_mimir_otlp.input]
}
}
otelcol.exporter.otlphttp "sender_to_mimir_otlp" {
client {
endpoint = "http://mimir-instance:3102/otlp"
tls { insecure = true }
}
}
prometheus.receive_http "receiver_classic_metrics" {
http {
listen_address = "0.0.0.0"
listen_port = 3102
}
forward_to = [prometheus.remote_write.sender_to_mimir.receiver,
prometheus.echo.received_metrics_exposure.receiver] // ajouter pour exposer les métriques reçues // à supprimer une fois migration prom vers Thanos
}
// echo permet d'écrire les métriques reçues sur le receive http sur /metrics
prometheus.echo "received_metrics_exposure" {
format = "openmetrics"
}
prometheus.scrape "echo_scraper" {
targets = [
{ __address__ = "received.internal", job = "receiver_debug"
},
{ __address__ = "echo.internal", job = "echo_debug" },
]
scrape_interval = "15s"
forward_to = [prometheus.remote_write.black_hole.receiver]
}
// remote write pour pourvoir scrapper les métriques reçues seulement // à supprimer après migration
prometheus.remote_write "black_hole" {
endpoint {
url = "localhost:0"
}
wal {
truncate_frequency = "1h" // nettoyage des wal toutes les x h
min_keepalive_time = "5m" // garde les fichiers au moins x min
max_keepalive_time = "1h" // garde au maximum x temps de backlog
}
}
// envoi vers mimir/thanos à commenter si pas d'envoi
prometheus.remote_write "sender_to_mimir" {
endpoint {
url = "http://mimir-instance:3102/api/v1/push"
}
wal {
truncate_frequency = "5h" // nettoyage des wal toutes les x h
min_keepalive_time = "5m" // garde les fichiers au moins x min
max_keepalive_time = "24h" // garde au maximum x temps de backlog
}
}
Logs
Tip
React with 👍 if this issue is important to you.