Support more OTEL gen-ai events & spans

[ion-elgreco]

Description:
I would like to integrate with other observability tools that map to all the OTEL gen-ai conventions. There is already support for 4 metrics, but would like to see spans and events as well.

[optional Relevant Links:]
>Any extra documentation required to understand the issue.

[mathetake]

i think this is reasonable. cc @codefromthecrypt as a subject matter expert

[codefromthecrypt]

ack. I think there are a couple things to consider.

1. spans envoy would make by default if tracing was enabled (this is easy, but not about genai)
2. spans that would be custom for envoy but representative of the gen-ai specs (includes contentious things like log events)

personally, I think the goal is not spans/events, rather a specific tool working that uses them. For example, Arize Phoenix (OSS) has an otel span format OpenInference (also OSS), which allows you to do evals. OpenInference also has practice around MCP which would likely become a feature here soon.

Conversely, the otel semantics are not widely adopted, and the log events api is in flux. For example, most eval solutions don't use any published otel semantic for this reason. Log events is being changed to logs (maybe). Some use older specs of span events, but that api is also in the process of being deprecated!

TL;DR; beyond normal envoy spans, I would suggest using openinference formatting as that solves one of the biggest obs issues in genai: evals.

[ion-elgreco]

@codefromthecrypt Interesting, wasn't aware of OpenInference, I see also some other servers like LangFuse supporting it. To me this makes sense, especially since you mention that OTEL is not as defined as I initially thought ^^

So that means we would have to support:

• spans
• traces

using these https://github.com/Arize-ai/openinference/blob/main/spec/semantic_conventions.md

[codefromthecrypt]

@ion-elgreco exactly right. I mean technically we would need something to emit the traces, so some plugin possibly wasm. It would need to treat buffering the req/resp as attributes carefully, for example with a hard cut-off size-wise in case someone puts the entire internet in their LLM context ;)

[ion-elgreco]

@codefromthecrypt ah ok, I don't have any experience yet with Go, but I do with Rust. I am planning to eventually work on this feature, I think it will be quite useful and might help with adoption

[mathetake]

#791