-
Notifications
You must be signed in to change notification settings - Fork 831
Allow metrics with the same name different labels #1800
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 16 commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
a85e35a
Type and label schema validation during registration
jaydeluca 58dddc8
start adding exposition merging
jaydeluca 5acc440
add some additional validation checks
jaydeluca c8eea69
optimize merge method
jaydeluca 43ab4b1
handle other exposition formats
jaydeluca 9289e4a
cleanups, ironing out edge cases
jaydeluca 6765715
fix linting
jaydeluca 004595f
merge main
jaydeluca 11efcfb
fix merge
jaydeluca 02c0db0
remove scrape time validation
jaydeluca 8b6bf20
cleanup registration
jaydeluca 54f5ecb
cleanup fallback
jaydeluca 74a46f3
add tests
jaydeluca dfa2114
test visibility
jaydeluca cbcd1ec
cleanup
jaydeluca 1e897dc
add otel sdk compatibility test
jaydeluca 707e4dc
make copys of labels, fix multicollector processing with try catch, f…
jaydeluca 73b47bd
fix histogram detection when merging snapshots, ensure created with t…
jaydeluca e3e0fe0
lint
jaydeluca 64d64d4
pr review
jaydeluca 89bac21
Merge branch 'main' into duplicate-names-registration-validation
jaydeluca e7aeef5
lint
jaydeluca b17cd0f
code review
jaydeluca 31aebc3
Merge branch 'main' into duplicate-names-registration-validation
jaydeluca 28a8514
proto update
jaydeluca 9bf69c8
be consistent about help/unit validation, ensure deregister labels
jaydeluca 6ebab1b
Merge branch 'main' into duplicate-names-registration-validation
jaydeluca 831645c
make exceptions more specific, undo unrelated changes to pom and gene…
jaydeluca d8490cd
add comments about use of RuntimeException:
jaydeluca File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -24,3 +24,5 @@ docs/public | |
| benchmark-results/ | ||
| benchmark-results.json | ||
| benchmark-output.log | ||
|
|
||
| *.DS_Store | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
59 changes: 59 additions & 0 deletions
59
integration-tests/it-exporter/it-exporter-duplicate-metrics-sample/pom.xml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
|
|
||
| <parent> | ||
| <groupId>io.prometheus</groupId> | ||
| <artifactId>it-exporter</artifactId> | ||
| <version>1.5.0-SNAPSHOT</version> | ||
| </parent> | ||
|
|
||
| <artifactId>it-exporter-duplicate-metrics-sample</artifactId> | ||
|
|
||
| <name>Integration Tests - Duplicate Metrics Sample</name> | ||
| <description> | ||
| HTTPServer Sample demonstrating duplicate metric names with different label sets | ||
| </description> | ||
|
|
||
| <dependencies> | ||
| <dependency> | ||
| <groupId>io.prometheus</groupId> | ||
| <artifactId>prometheus-metrics-exporter-httpserver</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.prometheus</groupId> | ||
| <artifactId>prometheus-metrics-core</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> | ||
| </dependencies> | ||
|
|
||
| <build> | ||
| <finalName>exporter-duplicate-metrics-sample</finalName> | ||
| <plugins> | ||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-shade-plugin</artifactId> | ||
| <executions> | ||
| <execution> | ||
| <phase>package</phase> | ||
| <goals> | ||
| <goal>shade</goal> | ||
| </goals> | ||
| <configuration> | ||
| <transformers> | ||
| <transformer | ||
| implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> | ||
| <mainClass> | ||
| io.prometheus.metrics.it.exporter.duplicatemetrics.DuplicateMetricsSample | ||
| </mainClass> | ||
| </transformer> | ||
| </transformers> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
| </project> |
91 changes: 91 additions & 0 deletions
91
.../main/java/io/prometheus/metrics/it/exporter/duplicatemetrics/DuplicateMetricsSample.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| package io.prometheus.metrics.it.exporter.duplicatemetrics; | ||
|
|
||
| import io.prometheus.metrics.core.metrics.Counter; | ||
| import io.prometheus.metrics.core.metrics.Gauge; | ||
| import io.prometheus.metrics.exporter.httpserver.HTTPServer; | ||
| import io.prometheus.metrics.model.snapshots.Unit; | ||
| import java.io.IOException; | ||
|
|
||
| /** Integration test sample demonstrating metrics with duplicate names but different label sets. */ | ||
| public class DuplicateMetricsSample { | ||
|
|
||
| public static void main(String[] args) throws IOException, InterruptedException { | ||
| if (args.length != 2) { | ||
| System.err.println("Usage: java -jar duplicate-metrics-sample.jar <port> <outcome>"); | ||
| System.err.println("Where outcome is \"success\" or \"error\"."); | ||
| System.exit(1); | ||
| } | ||
|
|
||
| int port = parsePortOrExit(args[0]); | ||
| String outcome = args[1]; | ||
| run(port, outcome); | ||
| } | ||
|
|
||
| private static void run(int port, String outcome) throws IOException, InterruptedException { | ||
| // Register multiple counters with the same Prometheus name "http_requests_total" | ||
| // but different label sets | ||
| Counter requestsSuccess = | ||
| Counter.builder() | ||
| .name("http_requests_total") | ||
| .help("Total HTTP requests by status") | ||
| .labelNames("status", "method") | ||
| .register(); | ||
| requestsSuccess.labelValues("success", "GET").inc(150); | ||
| requestsSuccess.labelValues("success", "POST").inc(45); | ||
|
|
||
| Counter requestsError = | ||
| Counter.builder() | ||
| .name("http_requests_total") | ||
| .help("Total HTTP requests by status") | ||
| .labelNames("status", "endpoint") | ||
| .register(); | ||
| requestsError.labelValues("error", "/api").inc(5); | ||
| requestsError.labelValues("error", "/health").inc(2); | ||
|
|
||
| // Register multiple gauges with the same Prometheus name "active_connections" | ||
| // but different label sets | ||
| Gauge connectionsByRegion = | ||
| Gauge.builder() | ||
| .name("active_connections") | ||
| .help("Active connections") | ||
| .labelNames("region", "protocol") | ||
| .register(); | ||
| connectionsByRegion.labelValues("us-east", "http").set(42); | ||
| connectionsByRegion.labelValues("us-west", "http").set(38); | ||
| connectionsByRegion.labelValues("eu-west", "https").set(55); | ||
|
|
||
| Gauge connectionsByPool = | ||
| Gauge.builder() | ||
| .name("active_connections") | ||
| .help("Active connections") | ||
| .labelNames("pool", "type") | ||
| .register(); | ||
| connectionsByPool.labelValues("primary", "read").set(30); | ||
| connectionsByPool.labelValues("replica", "write").set(10); | ||
|
|
||
| // Also add a regular metric without duplicates for reference | ||
| Counter uniqueMetric = | ||
| Counter.builder() | ||
| .name("unique_metric_total") | ||
| .help("A unique metric for reference") | ||
| .unit(Unit.BYTES) | ||
| .register(); | ||
| uniqueMetric.inc(1024); | ||
|
|
||
| HTTPServer server = HTTPServer.builder().port(port).buildAndStart(); | ||
|
|
||
| System.out.println( | ||
| "DuplicateMetricsSample listening on http://localhost:" + server.getPort() + "/metrics"); | ||
| Thread.currentThread().join(); // wait forever | ||
| } | ||
|
|
||
| private static int parsePortOrExit(String port) { | ||
| try { | ||
| return Integer.parseInt(port); | ||
| } catch (NumberFormatException e) { | ||
| System.err.println("\"" + port + "\": Invalid port number."); | ||
| System.exit(1); | ||
| } | ||
| return 0; // this won't happen | ||
| } | ||
| } |
181 changes: 181 additions & 0 deletions
181
...xporter-test/src/test/java/io/prometheus/metrics/it/exporter/test/DuplicateMetricsIT.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,181 @@ | ||
| package io.prometheus.metrics.it.exporter.test; | ||
|
|
||
| import static org.assertj.core.api.Assertions.assertThat; | ||
|
|
||
| import io.prometheus.client.it.common.ExporterTest; | ||
| import io.prometheus.metrics.expositionformats.generated.com_google_protobuf_4_33_4.Metrics; | ||
| import java.io.IOException; | ||
| import java.net.URISyntaxException; | ||
| import java.util.List; | ||
| import org.junit.jupiter.api.Test; | ||
|
|
||
| class DuplicateMetricsIT extends ExporterTest { | ||
|
|
||
| public DuplicateMetricsIT() throws IOException, URISyntaxException { | ||
| super("exporter-duplicate-metrics-sample"); | ||
| } | ||
|
|
||
| @Test | ||
| void testDuplicateMetricsInPrometheusTextFormat() throws IOException { | ||
| start(); | ||
| Response response = scrape("GET", ""); | ||
| assertThat(response.status).isEqualTo(200); | ||
| assertContentType( | ||
| "text/plain; version=0.0.4; charset=utf-8", response.getHeader("Content-Type")); | ||
|
|
||
| String expected = | ||
| """ | ||
| # HELP active_connections Active connections | ||
| # TYPE active_connections gauge | ||
| active_connections{pool="primary",type="read"} 30.0 | ||
| active_connections{pool="replica",type="write"} 10.0 | ||
| active_connections{protocol="http",region="us-east"} 42.0 | ||
| active_connections{protocol="http",region="us-west"} 38.0 | ||
| active_connections{protocol="https",region="eu-west"} 55.0 | ||
| # HELP http_requests_total Total HTTP requests by status | ||
| # TYPE http_requests_total counter | ||
| http_requests_total{endpoint="/api",status="error"} 5.0 | ||
| http_requests_total{endpoint="/health",status="error"} 2.0 | ||
| http_requests_total{method="GET",status="success"} 150.0 | ||
| http_requests_total{method="POST",status="success"} 45.0 | ||
| # HELP unique_metric_bytes_total A unique metric for reference | ||
| # TYPE unique_metric_bytes_total counter | ||
| unique_metric_bytes_total 1024.0 | ||
| """; | ||
|
|
||
| assertThat(response.stringBody()).isEqualTo(expected); | ||
| } | ||
|
|
||
| @Test | ||
| void testDuplicateMetricsInOpenMetricsTextFormat() throws IOException { | ||
| start(); | ||
| Response response = | ||
| scrape("GET", "", "Accept", "application/openmetrics-text; version=1.0.0; charset=utf-8"); | ||
| assertThat(response.status).isEqualTo(200); | ||
| assertContentType( | ||
| "application/openmetrics-text; version=1.0.0; charset=utf-8", | ||
| response.getHeader("Content-Type")); | ||
|
|
||
| // OpenMetrics format should have UNIT for unique_metric_bytes (base name without _total) | ||
| String expected = | ||
| """ | ||
| # TYPE active_connections gauge | ||
| # HELP active_connections Active connections | ||
| active_connections{pool="primary",type="read"} 30.0 | ||
| active_connections{pool="replica",type="write"} 10.0 | ||
| active_connections{protocol="http",region="us-east"} 42.0 | ||
| active_connections{protocol="http",region="us-west"} 38.0 | ||
| active_connections{protocol="https",region="eu-west"} 55.0 | ||
| # TYPE http_requests counter | ||
| # HELP http_requests Total HTTP requests by status | ||
| http_requests_total{endpoint="/api",status="error"} 5.0 | ||
| http_requests_total{endpoint="/health",status="error"} 2.0 | ||
| http_requests_total{method="GET",status="success"} 150.0 | ||
| http_requests_total{method="POST",status="success"} 45.0 | ||
| # TYPE unique_metric_bytes counter | ||
| # UNIT unique_metric_bytes bytes | ||
| # HELP unique_metric_bytes A unique metric for reference | ||
| unique_metric_bytes_total 1024.0 | ||
| # EOF | ||
| """; | ||
|
|
||
| assertThat(response.stringBody()).isEqualTo(expected); | ||
| } | ||
|
|
||
| @Test | ||
| void testDuplicateMetricsInPrometheusProtobufFormat() throws IOException { | ||
| start(); | ||
| Response response = | ||
| scrape( | ||
| "GET", | ||
| "", | ||
| "Accept", | ||
| "application/vnd.google.protobuf; proto=io.prometheus.client.MetricFamily;" | ||
| + " encoding=delimited"); | ||
| assertThat(response.status).isEqualTo(200); | ||
| assertContentType( | ||
| "application/vnd.google.protobuf; proto=io.prometheus.client.MetricFamily;" | ||
| + " encoding=delimited", | ||
| response.getHeader("Content-Type")); | ||
|
|
||
| List<Metrics.MetricFamily> metrics = response.protoBody(); | ||
|
|
||
| assertThat(metrics).hasSize(3); | ||
|
|
||
| // Metrics are sorted by name | ||
| assertThat(metrics.get(0).getName()).isEqualTo("active_connections"); | ||
| assertThat(metrics.get(1).getName()).isEqualTo("http_requests_total"); | ||
| assertThat(metrics.get(2).getName()).isEqualTo("unique_metric_bytes_total"); | ||
|
|
||
| // Verify active_connections has all 5 data points merged | ||
| Metrics.MetricFamily activeConnections = metrics.get(0); | ||
| assertThat(activeConnections.getType()).isEqualTo(Metrics.MetricType.GAUGE); | ||
| assertThat(activeConnections.getHelp()).isEqualTo("Active connections"); | ||
| assertThat(activeConnections.getMetricList()).hasSize(5); | ||
|
|
||
| // Verify http_requests_total has all 4 data points merged | ||
| Metrics.MetricFamily httpRequests = metrics.get(1); | ||
| assertThat(httpRequests.getType()).isEqualTo(Metrics.MetricType.COUNTER); | ||
| assertThat(httpRequests.getHelp()).isEqualTo("Total HTTP requests by status"); | ||
| assertThat(httpRequests.getMetricList()).hasSize(4); | ||
|
|
||
| // Verify each data point has the expected labels | ||
| boolean foundSuccessGet = false; | ||
| boolean foundSuccessPost = false; | ||
| boolean foundErrorApi = false; | ||
| boolean foundErrorHealth = false; | ||
|
|
||
| for (Metrics.Metric metric : httpRequests.getMetricList()) { | ||
| List<Metrics.LabelPair> labels = metric.getLabelList(); | ||
| if (hasLabel(labels, "status", "success") && hasLabel(labels, "method", "GET")) { | ||
| assertThat(metric.getCounter().getValue()).isEqualTo(150.0); | ||
| foundSuccessGet = true; | ||
| } else if (hasLabel(labels, "status", "success") && hasLabel(labels, "method", "POST")) { | ||
| assertThat(metric.getCounter().getValue()).isEqualTo(45.0); | ||
| foundSuccessPost = true; | ||
| } else if (hasLabel(labels, "status", "error") && hasLabel(labels, "endpoint", "/api")) { | ||
| assertThat(metric.getCounter().getValue()).isEqualTo(5.0); | ||
| foundErrorApi = true; | ||
| } else if (hasLabel(labels, "status", "error") && hasLabel(labels, "endpoint", "/health")) { | ||
| assertThat(metric.getCounter().getValue()).isEqualTo(2.0); | ||
| foundErrorHealth = true; | ||
| } | ||
| } | ||
|
|
||
| assertThat(foundSuccessGet).isTrue(); | ||
| assertThat(foundSuccessPost).isTrue(); | ||
| assertThat(foundErrorApi).isTrue(); | ||
| assertThat(foundErrorHealth).isTrue(); | ||
|
|
||
| Metrics.MetricFamily uniqueMetric = metrics.get(2); | ||
| assertThat(uniqueMetric.getType()).isEqualTo(Metrics.MetricType.COUNTER); | ||
| assertThat(uniqueMetric.getMetricList()).hasSize(1); | ||
| assertThat(uniqueMetric.getMetric(0).getCounter().getValue()).isEqualTo(1024.0); | ||
| } | ||
|
|
||
| @Test | ||
| void testDuplicateMetricsWithNameFilter() throws IOException { | ||
| start(); | ||
| // Only scrape http_requests_total | ||
| Response response = scrape("GET", nameParam()); | ||
| assertThat(response.status).isEqualTo(200); | ||
|
|
||
| String body = response.stringBody(); | ||
|
|
||
| assertThat(body) | ||
| .contains("http_requests_total{method=\"GET\",status=\"success\"} 150.0") | ||
| .contains("http_requests_total{endpoint=\"/api\",status=\"error\"} 5.0"); | ||
|
|
||
| // Should NOT contain active_connections or unique_metric_total | ||
| assertThat(body).doesNotContain("active_connections").doesNotContain("unique_metric_total"); | ||
| } | ||
|
|
||
| private boolean hasLabel(List<Metrics.LabelPair> labels, String name, String value) { | ||
| return labels.stream() | ||
| .anyMatch(label -> label.getName().equals(name) && label.getValue().equals(value)); | ||
| } | ||
|
|
||
| private String nameParam() { | ||
| return "name[]=" + "http_requests_total"; | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.