How does the framework determine tool calls based on model response format?

[RhythmicWave]

Question

I have a question regarding the framework's mechanism for tool invocation.

I've observed that the model doesn't always trigger an actual tool call, which may be related to its capabilities. To improve this, I want to guide the model via prompts on the correct way to trigger tools when it fails.

Could you please explain how the framework decides whether to perform a tool call based on the model's response format? Specifically:

• What specific structure, keywords, or format in the model's output does the framework use to identify a tool call?
• Are there any predefined patterns or rules for detecting valid tool invocations?
• How can we adjust prompts to ensure the model's responses align with the expected tool-call format?

Understanding this process will help in designing more effective prompts to reduce missed tool invocations. Thank you for your help!

Additional Context

No response

[DouweM]

@RhythmicWave Pydantic AI does not directly parse tool calls from model response text. Instead, the model APIs are responsible for parsing tool calls and returning them in a structured way that Pydantic AI will handle as implemented in the model classes.

What model/API are you using?

[RhythmicWave]

The Kimi-K2-Instruct-0905 model I am using sometimes outputs a CallToolsNode but only contains a TextPart without a ToolCallPart, so the actual tool is not invoked. Trying a few more times might lead to success, but it depends on luck.

When I switched to the DeepSeek-V3.1-Terminus model, it performed the tool invocation smoothly, containing both TextPart and ToolCallPart.
Both of the aforementioned models are wrapped using the OpenAIChatModel.

So I'd like to ask if there are any methods to enhance the model's ability to understand how to invoke tools?

[DouweM]

@RhythmicWave How are you running the models? Using something like Ollama or vLLM, or a cloud provider? Parsing the output to pull out tool calls (and prompting the model to produce the expected format) would be that provider's API's responsibility. Pydantic AI doesn't have a way to do this although there's an open issue: #2033. I'll mark this issue as a duplicate of that one, but feel free to keep asking questions here.