-
Notifications
You must be signed in to change notification settings - Fork 544
Description
Component(s)
prometheus.scrape
What's wrong?
When scraping an endpoint that returns application/openmetrics-text with blank lines between metric families, Alloy's OpenMetrics parser treats the blank line as an invalid token and marks the scrape as up=0, dropping all metrics from metric families after the first blank line.
The OpenMetrics specification states that blank lines should be ignored by parsers.
Steps to reproduce
Scrape any endpoint returning the following OpenMetrics response (note the blank line between metric families):
# HELP django_health_check_status Health check status (1 = healthy, 0 = unhealthy)
# TYPE django_health_check_status gauge
django_health_check_status{check="Database(alias='default')"} 1
# HELP django_health_check_response_time_seconds Health check response time in seconds
# TYPE django_health_check_response_time_seconds gauge
django_health_check_response_time_seconds{check="Database(alias='default')"} 0.014698
# HELP django_health_check_overall_status Overall health check status
# TYPE django_health_check_overall_status gauge
django_health_check_overall_status 1
# EOF
Expected behavior
All 3 metric families are parsed successfully, up=1.
Actual behavior
Parsing stops at the blank line. Only the first metric family is scraped. up=0.
The following error is visible in the Alloy UI target debug page under last_error:
expected a valid start token, got "\n" ("INVALID") while parsing: "\n"
Environment
Alloy version: grafana/alloy:latest [v1.13.1]
Deployment: Docker container
Scrape target: django-health-check library (https://github.com/codingjoe/django-health-check), which emits blank lines between metric families in its OpenMetrics renderer
System information
Docker Image Debian 13
Software version
grafana/alloy:latest image ( v1.13.1 )
Configuration
prometheus.scrape "django_health" {
targets = [{
__address__ = "django-app:8000",
}]
metrics_path = "/health/"
params = { "format" = ["openmetrics"] }
forward_to = [prometheus.remote_write.grafana_cloud.receiver]
scrape_interval = "15s"
job_name = "django_health"
}
prometheus.remote_write "grafana_cloud" {
endpoint {
url = #######
basic_auth {
username = ######
password = ######
}
}
}
Logs
Tip
React with π if this issue is important to you.