Support (auto) instrument Spring 7 interface declarative HTTP clients

[holgerstolzenberg]

Is your feature request related to a problem? Please describe.

I am currently integrating "opentelemetry-spring-boot-starter" with a Spring Boot v4 application.
Whereas nearly everything worked easily OOTB, I had some struggles getting a Spring Framework 7 interface based HTTP client instrumented properly.

In order to solve the issue, I ended up adding a custom interceptor implementation. One smaller obstacle was the fact that for whatever reason I cannot import classes like SpringWebMvcTelemetry.

I'll provide the bean definition of the client and the custom interceptor here, so others can use that in the mean time.

public interface AuditEventClient {
  @WithSpan
  @PostExchange(url = "/audit-events", contentType = AuditEvent.MIME_TYPE)
  void pushEvent(
      @RequestParam @SpanAttribute("audit.event.mode") final Mode mode,
      @RequestBody @SpanAttribute("audit.event") final AuditEvent event);
}

@Configuration
class AuditTrailConfiguration {
  private static final String INTERNAL_CLIENT = "internal-client";

  @Bean
  AuditEventClient auditEventClient(
      final ClientRegistrationRepository repository,
      final OAuth2AuthorizedClientService service,
      final RestClient.Builder builder,
      final AuditTrailConfigurationProperties properties,
      final ObjectProvider<OpenTelemetry> openTelemetryProvider) {
    val authorizedClientManager =
        new AuthorizedClientServiceOAuth2AuthorizedClientManager(repository, service);

    val oauthHttpRequestInterceptor =
        new OAuth2ClientHttpRequestInterceptor(authorizedClientManager);
    oauthHttpRequestInterceptor.setClientRegistrationIdResolver(_ -> INTERNAL_CLIENT);

    val openTelemetry = openTelemetryProvider.getIfAvailable();
    if (isNull(openTelemetry)) {
      throw new IllegalStateException("OpenTelemetry is not available");
    }

    val baseUrl = properties.client().baseUrl();
    val restClient =
        builder
            .baseUrl(baseUrl)
            .requestInterceptor(OpenTelemetryClientHttpRequestInterceptor.with(openTelemetry)) // ‼️ manual registration necessary
            .requestInterceptor(oauthHttpRequestInterceptor)
            .build();
    val exchangeAdapter = RestClientAdapter.create(restClient);

    return HttpServiceProxyFactory.builderFor(exchangeAdapter)
        .build()
        .createClient(AuditEventClient.class);
  }
}

// custom implementation, maybe there is a re-usable one already?
@UtilityClass
public final class OpenTelemetryClientHttpRequestInterceptor {

  @Nonnull
  public static ClientHttpRequestInterceptor with(@Nonnull final OpenTelemetry openTelemetry) {
    checkNotNull(openTelemetry, "openTelemetry");

    return (request, body, execution) -> {
      openTelemetry
          .getPropagators()
          .getTextMapPropagator()
          .inject(
              Context.current(), request, (req, key, value) -> req.getHeaders().set(key, value));
      return execution.execute(request, body);
    };
  }
}

Describe the solution you'd like

It would be cool if the spring boot starter just automatically instruments these clients as it instruments RestClient, WebClient etc.

Describe alternatives you've considered

• I tried Arconia Framework but had other issues.
• I tried new spring-boot-starter-telemetry but it looks like if currently lacks an option to push metrics via grpc

Additional context

None.

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[laurit]

@holgerstolzenberg could you provide a minimal application that reproduces the issue along with any necessary instructions. In your sample code the interface based client uses RestClient.Builder and in the test that I tried that is automatically instrumented.

[holgerstolzenberg]

@laurit okay so I have put together a small demo project:
https://github.com/holgerstolzenberg/opentelemetry-issue-16412

It has a quite large dependency set and a Docker compose stack. I have added some Spring Configuration classes, that I thought could have impact. Hope that helps.

During the build-up I realised that my initial request might point into the wrong direction. The test project reveals that indeed, as expected, the client gets instrumented.

Yet there is still an issue for me. I guess my initial finding somehow got cluttered during the initial integration in my project. If the provided interceptor is not used, the client will get a new trace_id not inheriting the trace from the surrounding service call. This then causes partial non connected flows when viewing the traces in Grafana Tempo.

I would have expected the client call to inherit the trace id, especially because @WithSpan is implicitly inherit=true.
Am I overlooking something? Is that the expected behaviour?

The demo project contains instructions in the README.md on how to "reproduce" the issue/behaviour.