Allow running agent with `ModelResponse` as last message

[fortzi]

Description

Sometimes it's useful to have an assistant message be the last message in the message history.
For example, you have a chat bot that sends another message after some time that the user doesn't respond:

User: Hi can I get help with X?
Agent: Sure, can you provide detail Y?
<some time passes>
Agent: If you don't have this detail I can do Z for you

Since Agent.run() (and Agent.iter()) only get a user prompt as an input, this is impossible.
Previously, it was possible to put some dummy prompt in there, and remove it in message history processing. Since v1.0.8 this is no longer possible due to a new validation in pydantic_ai._agent_graph._process_message_history that raises exceptions.UserError('Processed history must end with a ModelRequest.')

Is this validation mission critical, or is mostly for detecting probably-wrong history processing?

Maybe this validation can be opted out of? WDYT?

References

No response

[pydanticai-bot]

PydanticAI Github Bot Found 1 issues similar to this one:

1. "Forbid first ModelResponse message #2992" (90% similar)

[phemmer]

Not a maintainer, just chiming in with my own $0.02, but I don't think this would make much sense. If you send a conversation history to an LLM where the LLM is the last response, how is the LLM supposed to know what to do with that? Like as far as it can see, it already responded. You would have to add an additional message to the end to give it the information needed to understand why and how it should add a new response. In this case, it would be something like "The user has not responded in X minutes. You should consider whether they need additional assistance".

And yes, some models do break if you give them a sequence of messages types that doesn't make sense. Some providers won't even invoke the model and will just throw an error at you.

Reading this statement, I wonder if there might be a misunderstanding:

Since Agent.run() (and Agent.iter()) only get a user prompt as an input, this is impossible.

Agent.run() can accept things other than a simple user message (prompt). In my own use, I don't use that functionality at all. I tend to maintain conversation history myself, and send it an entire conversation. agent.run(message_history=...).

[fortzi]

@phemmer the instructions (kind of) separate from the actual conversation, in the sense that pydantic-ai stitches it as system message to the beginning of the conversation (see _map_messages).
For example it would say "You are a support agent working for an X company, be nice, here are some rules..." etc.
The conversation itself doesn't include an instruction for each run.

Consider another use case for this:

T_1: User sends message_1
T_2: Agent receives and processes message_1
T_3: Agent sends message_2
T_4: User sends message_3
T_5: User receives message_2

Next thing that happens, the agent needs to respond to the following conversation:

User: message_1
User: message_2
Agent: message_3

As of now, this isn't supported by pydantic-ai, despite it representing the conversation as the customer perceives it.

To address your note about the LLM not knowing what to do, well, it always says something. With the right instructions telling it its job, it happily continues ("completes") the conversation from wherever it's asked to, at least OpenAI does.

[fortzi]

Just found #2778 (comment), which solves the problem. Closing this for now

[phemmer]

the instructions (kind of) separate from the actual conversation, in the sense that pydantic-ai stitches it as system message to the beginning of the conversation

Instructions are nothing more than a system message. And like the user message I explained earlier, you are not forced to use the instructions parameter to provide them. You are able to use the message_history parameter of agent.run() to build your own message history, including instructions (system messages), and you can put those instructions anywhere in the history you want. They do not have to be at the beginning.

[fortzi]

That's true, I can fabricate the message history myself, but that adds complexity. Since the problem I described is a fact of life when building chat applications, it would make sense that the recommended usage pattern of the library supports it (which it fortunately does).