Skip to content

[Feature] Expose prometheus.receive_http received metrics #5667

@lekmitiralf

Description

@lekmitiralf

Component(s)

prometheus.receive_http

What's wrong?

Problem:
Metrics received by prometheus.receive_http (ex: prometheus_receive_http_receiver_classic_metrics_received_samples_total) are NOT exposed on Alloy's standard /metrics endpoint, blocking external monitoring.
Impact:
Impossible monitoring of proxy/gateway metrics without local TSDB (Mimir/Prometheus).

v0 API endpoints are internal debug only, not scrapable by external Prometheus.

Use Case:
Get recieved metrics from http through the UI
Or, get recieved metrics through a prometheus.echo that output in the /metrics or /api/v0/component/prometheus.echo./metrics
Or get metrics from a scraper

Steps to reproduce

use Alloy as a proxy.
Create a http receiver and send metrics from alloy agent to alloy proxy gateway
❌/api/v0/component/prometheus.receive_http.receiver_classic_metrics/metrics → 404
❌ /api/v0/component/prometheus.echo.received_metrics_exposure/metrics → 404
❌ /api/v0/component/prometheus.scrape.echo_scraper/metrics → 404
❌ no received metrics in the /metrics of alloy UI

System information

Linux 3.10.0-1160.145.1.el7.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux

Software version

Grafana Alloy 1.13.2

Configuration

// activer le mode debug pour afficher toutes les métriques passantes par le proxy
livedebugging {
  enabled = true
}

// reception des métriques de traces 
otelcol.receiver.otlp "receiver_traces_metrics" {
        grpc {
	  endpoint = "0.0.0.0:4319"
	}
	http {
	  endpoint = "0.0.0.0:4320"
	}
	output {
	  metrics = [otelcol.exporter.otlphttp.sender_to_mimir_otlp.input] 
	}
}

otelcol.exporter.otlphttp "sender_to_mimir_otlp" {
   client {
     endpoint = "http://mimir-instance:3102/otlp"
     tls { insecure = true }
}
}

prometheus.receive_http "receiver_classic_metrics" {
  http {
    listen_address = "0.0.0.0"
    listen_port    = 3102
  }
  forward_to = [prometheus.remote_write.sender_to_mimir.receiver,
  prometheus.echo.received_metrics_exposure.receiver]  // ajouter pour exposer les métriques reçues // à supprimer une fois migration prom vers Thanos
}
// echo permet d'écrire les métriques reçues sur le receive http sur /metrics
prometheus.echo "received_metrics_exposure" { 
  format = "openmetrics"  
}
prometheus.scrape "echo_scraper" {
  targets = [
    { __address__ = "received.internal", job = "receiver_debug"
       },
    { __address__ = "echo.internal", job = "echo_debug" },
  ]
  scrape_interval = "15s"
  forward_to = [prometheus.remote_write.black_hole.receiver]
}

// remote write pour pourvoir scrapper les métriques reçues seulement // à supprimer après migration
prometheus.remote_write "black_hole" {
        endpoint {
                url = "localhost:0"                                                                      
        }
          
        wal {
        truncate_frequency = "1h" // nettoyage des wal toutes les x h
        min_keepalive_time = "5m"  // garde les fichiers au moins x min
        max_keepalive_time = "1h"  // garde au maximum x temps de backlog
        }
}

// envoi vers mimir/thanos à commenter si pas d'envoi
prometheus.remote_write "sender_to_mimir" {
        endpoint {
                url = "http://mimir-instance:3102/api/v1/push"
  
        }

        wal {
        truncate_frequency = "5h" // nettoyage des wal toutes les x h
        min_keepalive_time = "5m"  // garde les fichiers au moins x min
        max_keepalive_time = "24h"  // garde au maximum x temps de backlog
        }
}

Logs


Tip

React with 👍 if this issue is important to you.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions