[WIP] Adapt to the latest input and output semantic specifications

[Cirilla-zmh]

No description provided.

[otelbot-java-instrumentation]

🔧 The result from spotlessApply was committed to the PR branch.

[Cirilla-zmh]

jackson annotations or io.opentelemetry.api.common.Value?

[Cirilla-zmh]

If possible, abstracting all StreamListener types into a single interface might be a better choice. It should have an onChunk(OutputMessages) method, an onEnd() method, and an onError(Throwable) method. We can complete the common collection logic here uniformly — such as the aggregation of OutputMessages, the captures of usage tokens and time per output chunk , etc.

How do you think about that? cc @anuraaga

[Cirilla-zmh]

By combining this instance into the GenAiAttributesExtractor, we can uniformly manage various collection switches in the AttributesExtractor — whether to collect chat history, whether to collect as span attributes or as logs, etc.

However, this also means that I might need to emit events in the AttributesExtractor, and I'm not sure if this is the right approach cc @trask. But according to the current semantic specification, we need to record some necessary attributes in events — and these attributes can almost only be obtained in the AttributesExtractor and SpanProcessor.

If we don't do this in the AttributesExtractor, the best approach might be to define an EventsExtractor in the Instrumenter and allow it to read the current attributes like:

public interface EventsExtractor {
  public Context onStart(Attributes startAttributes, Context context);
  public void onEnd(Attributes endAttributes, Context context);
}

[Cirilla-zmh]

OperationListener maybe another nice choice.

[Cirilla-zmh]

Each choice has a finish_reason, so which one should be recorded in the span's gen_ai.response.finish_reasons at this point?

[Cirilla-zmh]

Should these parts be recorded in the same ChatMessage?

In multi-turn conversation scenarios, there may be multiple 'assistant messages' in the 'input messages' - should all of them be recorded? (I think we should truthfully record the input of each call, regardless of whether it's a multi-turn conversation or not.)

[Cirilla-zmh]

So far, spring-ai has provided proxies for many model service SDKs, such as openai, gemini, etc. Some of these may already have dedicated instrumentation, and if spring-ai instrumentation is enabled, it will generate two duplicate inference spans. Therefore, we have two choices:

• Do not implement spring-ai instrumentation at the model layer, relying entirely on the SDK's native instrumentation. However, some additional information may not be accessible (such as metadata defined in spring ai).

• Similar to spring-kafka, also implement instrumentation and add a ProcessTracingState. Suppress the SDK's native instrumentation through context:

opentelemetry-java-instrumentation/instrumentation/kafka/kafka-clients/kafka-clients-0.11/bootstrap/src/main/java/io/opentelemetry/javaagent/bootstrap/kafka/KafkaClientsConsumerProcessTracing.java

Lines 13 to 31 in 5307e20

	// have separate copies of helper classes.
	public final class KafkaClientsConsumerProcessTracing {
	private static final ThreadLocal<Boolean> wrappingEnabled = ThreadLocal.withInitial(() -> true);
	private KafkaClientsConsumerProcessTracing() {}
	public static boolean setEnabled(boolean enabled) {
	boolean previous = wrappingEnabled.get();
	wrappingEnabled.set(enabled);
	return previous;
	}
	public static boolean wrappingEnabled() {
	return wrappingEnabled.get();
	}
	public static BooleanSupplier wrappingEnabledSupplier() {
	return KafkaClientsConsumerProcessTracing::wrappingEnabled;
	}

[Cirilla-zmh]

see #15064

[anuraaga]

Really sorry for the delay, the notification made me realize I had a pending comment that never got sent. Basically the approach looks fine as long as Jackson is acceptable in the instrumentation API.

[anuraaga]

I believe this would need to be implementation because the API doesn't imply using the javaagent. The extra dependency I guess would be a concern with doing JSON marshaling within the instrumentation API itself, though when conventions rely on JSON, maybe it's unavoidable

[Cirilla-zmh]

Yes. It may simplify the construction by defining these structured messages with json properties, which can also be seen in python-instrumentation: https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/util/opentelemetry-util-genai/src/opentelemetry/util/genai/types.py

Here I use compileOnly because openai has depended on it yet, I just want to show my intention. However, implementation may lead to dependency conflicts. Maybe we should define these messages object in 'java-tooling' module, which has a shaded jackson dependency.