Support for TOON

[upman]

Description

TOON is a new serialization format that uses fewer tokens. Could we have an optional TOON format mode in pydantic-ai agents for both input and output tokens?

https://github.com/toon-format/toon
https://github.com/toon-format/toon-python

References

No response

[DouweM]

@upman Where would you expect Pydantic AI to use this? The only place we currently serialize data that's sent to the model is tool call results, which many models explicitly require to be JSON or a JSON string.

[JayGhiya]

yea as @DouweM it makes sense to use in user prompts where we provide context pydantic ai cannot do this as of now considering all model providers and models pivot to this.

[upman]

@DouweM Yes, tool calls results would be the place for it.

I thought more about how to have output in the TOON format and we would need to add the format's spec to the prompt. I don't think that is worth it currently.

I would suggest to have tool call results in TOON as an experimental feature behind a feature flag. The benchmarks show at par or better performance than JSON at retrieval tasks: https://github.com/toon-format/toon?tab=readme-ov-file#retrieval-accuracy

But having it as an option in pydantic-ai can help folks run their own benchmarks with their existing pydantic-ai agents.

[DouweM]

@upman I had a look at the big 3 models:

• Anthropic SDK accepts arbitrary string tool results without specifying it has to be JSON
• OpenAI Chat and Responses SDKs/APIs both accept strings, but the Responses API docs state it should be a JSON string
• Google SDK only accepts a JSON dict

I doubt those JSON requirements are very strict though; I'm sure Gemini would also work fine if we provide a dict of {'result': '<TOON string>'}, and I doubt the Responses API will error if the string is not actually JSON (and if it does, we can do what we'd do for Gemini).

This would need to be implemented in the ToolReturnPart.model_response_{str,object}() methods, which could gain a format: Literal['json', 'xml', 'toon'] = 'json' argument (with xml included for good measure because we already have a format_as_xml helper).

We don't currently have experimental flags or global flags of any kind, so I suggest we add a new ModelProfile.tool_result_format option for this that the model classes can pass to those ToolReturnPart.model_response_{str,object}() methods.

That way you can specify this on a per-model/agent basis and only use it where it's proven to work well. We could eventually make it the default for some or all models (although likely not until v2 at the earliest).

Would that work for you? Would you be up for submitting a PR?