Spring Boot starter: auto-configure Kafka metrics reporter

[drazzib]

Is your feature request related to a problem? Please describe.

I'm not sure if the current behavior is desired or if it's an oversight
(I'll let you reclassify this as a bug at your convenience).

On a Spring Boot service, we have just migrated from instrumentation via the OpenTelemetry JVM agent to using the Spring Boot Starter JAR as dependency. Unfortunately, with Spring Boot Starter auto-configuration, we no longer have Kafka metrics associated with our consumer or producer.

The metrics we used to depends on are described in this page: Library instrumentation for Kafka Clients#Usage (Metrics).

Describe the solution you'd like

If I understood correctly, the Kafka metrics are activated via KafkaMetricsUtil#enhanceConfig in the following javaagent instrumentations:

• opentelemetry-java-instrumentation/instrumentation/kafka/kafka-clients/kafka-clients-0.11/javaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/kafkaclients/v0_11/metrics/KafkaMetricsConsumerInstrumentation.java

  Line 63 in 9865c17

  enhanceConfig(config);

• opentelemetry-java-instrumentation/instrumentation/kafka/kafka-clients/kafka-clients-0.11/javaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/kafkaclients/v0_11/metrics/KafkaMetricsProducerInstrumentation.java

  Line 63 in 9865c17

  enhanceConfig(config);

It seems relevant - to me at least :) - to have something similar in the Spring Boot Starter module (perhaps enable/disable via a configuration property)

Describe alternatives you've considered

I don't see a way to add the properties (to enable OpenTelemetryMetricsReporter ) in Spring Kafka consumers or producers in the auto-configuration mechanisms (either KafkaInstrumentationAutoConfiguration or ConcurrentKafkaListenerContainerFactoryPostProcessor)

Additional context

Current behavior tested with:

Spring Boot: 3.3.6 / OpenTelemetry Spring Boot Starter: v2.7.0

[steverao]

cc @zeitlinger @jeanbisutti

[zeitlinger]

The spring starter can only use library instrumentations.

Kafka currently only has agent based instrumentation. I'm not sure if it can be done. If yes, a PR would be welcome.

[trask]

Kafka currently only has agent based instrumentation.

@zeitlinger check out

• https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/spring/spring-kafka-2.7/library
• https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/kafka/kafka-clients/kafka-clients-2.6/library

[zeitlinger]

Thx - missed that.

So the starter uses the spring library instrumentation, which doesn't support metrics - and spring doesn't seem to allow access to the underlying hooks.
Micrometer seems to create metrics from the operation interceptors - which we could also do here:

https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/spring/spring-boot-autoconfigure/src/main/java/io/opentelemetry/instrumentation/spring/autoconfigure/internal/instrumentation/kafka/KafkaInstrumentationAutoConfiguration.java

[laurit]

I don't see a way to add the properties (to enable OpenTelemetryMetricsReporter ) in Spring Kafka consumers or producers in the auto-configuration mechanisms (either KafkaInstrumentationAutoConfiguration or ConcurrentKafkaListenerContainerFactoryPostProcessor)

I'd try DefaultKafkaConsumerFactoryCustomizer and DefaultKafkaProducerFactoryCustomizer updateConfigs should let you update the configuration.