Show system instructions for pydantic-ai LLM panel

[oliviaBahr]

Description

Right now when you instrument with @SomeAgent.system_prompt you get a beautifully rendered visual, but when you instrument with @SomeAgent.instructions the instructions are invisible.

It would be great to see agent instructions included in the LLM panel.

[alexmojaki]

@DouweM @Kludex this is similar to the problem of recording tool definitions on agent spans. How do we record dynamic instructions? Can they change at each step so it doesn't really make sense?

[DouweM]

Can they change at each step so it doesn't really make sense?

@alexmojaki Yep, they're regenerated at every step based on the run context. Maybe we should show it at the start, and then if we detect that the text changed, repeat the changed version with some special styling ahead of the new user message?

[oliviaBahr]

Maybe we should show it at the start, and then if we detect that the text changed, repeat the changed version with some special styling ahead of the new user message?

I would love this as a configuration option to display dynamic system things only on change or at every step. Especially for getting started configuring an agent, it can be really nice to see every single input/output for the model to keep track of ctx etc. without needing to log manually.

The reason I initially put this issue in Logfire is because I thought all of this data was already being recorded with otel and it was simply a matter of displaying it in the web app. It seems I'm mistaken here so I'd appreciate if someone could point me to where exactly in the codebase this is taking place.

[DouweM]

@oliviaBahr This (and the referenced method) is where we read the instructions from the most recent ModelRequest in message history, and send it in the gen_ai.system_instructions OTel attribute:

pydantic-ai/pydantic_ai_slim/pydantic_ai/models/instrumented.py

Lines 253 to 254 in ff0fbe3

	instructions = InstrumentedModel._get_instructions(input_messages) # pyright: ignore [reportPrivateUsage]
	system_instructions_attributes = self.system_instructions_attributes(instructions)

That attribute is stored on each per-LLM-request chat ... span, whereas @agent.system_prompt messages get their own role=system message, so it's possible that Logfire is currently only showing the latter, not the former.

If we show the last chat span's history on the agent run span detail pane, we'd only get to see the final instructions, not the ones that were active when each response in the history was generated, so we may need to do some merging of chat spans' histories to show the instructions at each step.

[dmontagu]

I have a proposal for how to fix this in #3131 that basically just always uses the last request's instructions in the agent run span, but also adds an attribute when there are different instructions in different requests as a way to signal to you that you may want to actually dig into the specific instructions for each step by reviewing the individual LLM request spans.