1.17 Migration Guide

Prometheus Registry

Micrometer 1.17.0 upgrades to the Prometheus Java client 1.5.0, which includes some behavioral changes. Additionally, we have removed the restriction that meters having the same name must all have the same set of tag names - see https://github.com/micrometer-metrics/micrometer/issues/877.

Duplicate Prometheus names

Meters with distinct names in Micrometer can ultimately map to the same Prometheus name due to the rules around suffixes in Prometheus. The following is an example of that because, in Prometheus, Counters always get the _total suffix added to them.

registry.counter("test").increment();
registry.counter("test.total").increment(42);

Prior to Micrometer 1.17.0, only the second meter would be output in the Prometheus scrape with no error to indicate a problem in the instrumentation.

Since Micrometer 1.17.0, the above code will keep only the first meter and subsequent meters that are effectively duplicates (due to mapping to the same Prometheus name) will fail to register. You can control the behavior of what happens when a meter fails to register with registry.config().onMeterRegistrationFailed(...). By default in the Prometheus registry, a warn-then-debug logger will log the failure. For the above sample code, you would get the following log message:

The meter (MeterId{name='test.total', tags=[]}) registration has failed: A meter with the same Prometheus name (test) is already registered. Registering this meter with a different Micrometer name that maps to the same Prometheus name would fail with an exception on scrape. Note that subsequent logs will be logged at debug level.

Duplicate time series from different Meters

There are other ways in which things can clash and MUST be avoided in instrumentation. An example is the following code:

Timer.builder("test").register(registry).record(Duration.ofMillis(10));
Gauge.builder("test_seconds_max", () -> 42).register(registry);

There is a conflict because the Timer has base units of seconds and will produce a max that clashes with the Gauge. This will fail on scrape, which is existing behavior, but the exception thrown has changed due to changes in the Prometheus Java client.

• Before: java.lang.IllegalStateException: test_seconds_max: duplicate metric name.
• After: io.prometheus.metrics.model.snapshots.DuplicateLabelsException: Duplicate labels for metric "test_seconds_max": {}

In the future, we may try to improve this by failing to register the meter that will result in duplication on scrape, but instrumentation MUST avoid registering meters where this will happen if there is a chance your users will target Prometheus as a backend.