How to handle multi-turn conversation evals with Pydantic AI Evals

[xjose97x]

Question

Hey everyone 👋

I’m trying to figure out how to handle multi-turn conversations in Pydantic AI Evals.

Are there any examples or docs that demonstrate this? I’d love to see how others are approaching it.

Thanks a lot!

Additional Context

No response

[DouweM]

@xjose97x You can have your evaluated function return the entire run result rather than just the output, and then use custom evaluators that check result.all_messages(), result.response etc. Does that help?

[xjose97x]

@xjose97x You can have your evaluated function return the entire run result rather than just the output, and then use custom evaluators that check result.all_messages(), result.response etc. Does that help?

Yeah, I think I'll follow that approach.
Thanks for the tip, @DouweM !