Add `tool_choice` setting #3611

dsfaccini · 2025-12-02T03:24:33Z

Adds tool_choice to ModelSettings, letting users control how the model interacts with function tools.

Currently Pydantic AI internally decides whether to use tool_choice='auto' or 'required' based on output configuration, but users have no way to override this. The workaround was using extra_body={'tool_choice': 'none'} which is provider-specific and doesn't work everywhere.

This PR allows the user to set tool_choice to:

'auto' - model decides whether to call tools
'required' - model must call a tool
'none' - model can't use function tools
['tool_a', 'tool_b'] - model must use one of these specific tools

One important distinction: this only affects function tools (the ones you register on the agent), not output tools (used internally for structured output). So if you have an agent with output_type=SomeModel
and you set tool_choice='none', the output tool stays available - you'll just get a warning about it.

Implementation is spread across all model providers since each has its own API format for tool_choice.
Added a resolve_tool_choice utility that handles validation (checking tool names exist, warning about
conflicts with output tools) and returns a normalized representation that each provider then maps to their specific format.

Bedrock is a bit of a special case - it doesn't support 'none' at all, so we fall back to 'auto' with a warning. Anthropic has a constraint where 'required' and specific tool selection don't work with thinking/extended thinking enabled.

TODO

document this somewhere in the docs

Clarify that agents can be produced by a factory function if preferred.

- pending: centralize tests?

dsfaccini · 2025-12-09T02:45:12Z

haven't integrated reviewed the code with #1820 in mind, so putting this back to draft

Resolve conflict in tests/models/test_anthropic.py by keeping both: - Container tests from upstream (container_id feature) - Tool choice tests from our branch 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

…rarily across the model run

The docstring examples in settings.py use RunContext without importing it, which fails ruff linting. Added test="skip" lint="skip" to skip these examples in the test suite. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

DouweM · 2025-12-09T20:07:33Z

docs/tools-advanced.md


 If both per-tool `prepare` and agent-wide `prepare_tools` are used, the per-tool `prepare` is applied first to each tool, and then `prepare_tools` is called with the resulting list of tool definitions.

+## Forcing Tool Use on First Request {#force-first-request}


Please split this feature out into a separate PR; we want to keep it in mind as we implement model_settings['tool_choice'], but they're separate features

DouweM · 2025-12-09T20:10:06Z

docs/tools-advanced.md

+
+A common pattern is to force the model to use specific tools on the first request before allowing it to respond freely. This is particularly useful for RAG (Retrieval Augmented Generation) patterns where you want the model to search or retrieve information before answering.
+
+Use the `force_first_request` parameter on `@agent.tool` or `@agent.tool_plain` to force the model to use specific tools on the first model request of each [`run()`][pydantic_ai.Agent.run]:


We can discuss this in the followup PR for this feature, but I'll note that this does not follow my proposal in #2799 (comment):

I'm thinking we could have tool_choice on ModelSettings, and then add a hook to let the entire ModelSettings be customized on a per-step basis. But for the very common case of "require tool X to be used before generating output", it'd be nice to have a dedicated argument on agent.run that doesn't require writing a custom callback that checks ctx.run_step etc. So I was thinking of also adding a agent.run(..., require_tool_use=...) that can take True, a str tool name, a Sequence[str] of tool names, or (default) False. Would that be sufficient for your use case?

This currently approach is too inflexible and not centralized enough.

DouweM · 2025-12-09T20:10:50Z

docs/tools-advanced.md

+
+## Dynamic Tool Choice {#dynamic-tool-choice}
+
+For more complex scenarios, you can provide a callable for `tool_choice` in [`model_settings`][pydantic_ai.settings.ModelSettings] that dynamically determines which tools the model can use at each step:


We should link to the model settings docs in agents.md (I believe)

DouweM · 2025-12-09T20:11:50Z

docs/tools-advanced.md

+
+result = agent.run_sync(
+    'Find information about Python and summarize it.',
+    model_settings={'tool_choice': my_tool_choice},


Model settings need to be serializable, so this won't work.

Please follow my suggestion from the issue to add a new hook to let the entire model settings be "prepared"/modified ahead of each run step, not just this specific field.

DouweM · 2025-12-09T20:12:28Z

docs/tools-advanced.md

+
+- `None` — Use default PydanticAI behavior (auto/required based on output type)
+- `'auto'` — Model decides whether to use tools
+- `'required'` — Model must use a tool


Since the function/output distinction is important here, we should clarify that this means the model must use a function tool

DouweM · 2025-12-09T20:48:03Z

pydantic_ai_slim/pydantic_ai/models/groq.py

+                    type='function',
+                    function={'name': resolved.tool_names[0]},
+                )
+            warnings.warn(


Same as above, no warnings please

DouweM · 2025-12-09T20:48:22Z

pydantic_ai_slim/pydantic_ai/models/groq.py

+            return 'auto'
+
+        if resolved.mode in ('auto', 'required'):
+            return resolved.mode


See above; we do actually need to handle these in a special way, not just pass them on

DouweM · 2025-12-09T20:49:40Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+
+    if resolved.mode == 'required':
+        if not openai_profile.openai_supports_tool_choice_required:
+            warnings.warn(


No warning please

We should be sending a list of function tool names, so the profile setting may not be relevant anyway

DouweM · 2025-12-09T20:51:09Z

pydantic_ai_slim/pydantic_ai/models/openai.py



+@dataclass
+class _OpenAIToolChoiceResult:


I think having 2 classes for intermediate representations of tool choice is too much; I'd rather have the logic inline like we do in the other model classes.

DouweM · 2025-12-09T20:52:02Z

pydantic_ai_slim/pydantic_ai/settings.py

+    * `'none'`: Model cannot use function tools (output tools remain available if needed)
+    * `list[str]`: Model must use one of the specified function tools (validated against registered tools)
+
+    **Dynamic callable:**


As mentioned, we can't support this in this way

dsfaccini added 9 commits November 28, 2025 12:42

Clarify usage of agent factories

8d52d65

Clarify that agents can be produced by a factory function if preferred.

Merge branch 'pydantic:main' into main

3128b4a

implement tool choice resolution per model

6d942f5

- centralize logic in utility and add tests for all providers

0585347

- pending: centralize tests?

coverage?

96681ac

coverage

e71dc86

imrpove tests

5c387fd

Merge branch 'main' into tool-choice

4dcfbe4

improvde code quality

363c718

dsfaccini mentioned this pull request Dec 5, 2025

[Bedrock] ValidationException: "Thinking may not be enabled when tool_choice forces tool use" with interleaved thinking on Sonnet 4.5 #3092

Open

2 tasks

dsfaccini added 7 commits December 5, 2025 13:59

deduplicate openai logic

31bb4e1

remove cast

338a073

re-run existent cassettes and record new ones for new tool choice tests

07fcb6b

fix snapshots

51cada5

fix tests

6597e0b

Merge branch 'main' into tool-choice

914748f

upgrade to newer models

57ff6bf

dsfaccini marked this pull request as ready for review December 9, 2025 01:53

dsfaccini marked this pull request as draft December 9, 2025 02:44

dsfaccini and others added 3 commits December 9, 2025 09:34

support tool choice callable to force tools on first request or arbit…

1a46a7b

…rarily across the model run

Merge branch 'main' into tool-choice

5e4cfb7

dsfaccini marked this pull request as ready for review December 9, 2025 18:06

dsfaccini and others added 2 commits December 9, 2025 13:19

skip tests

70dc917

DouweM requested changes Dec 9, 2025

View reviewed changes

DouweM self-assigned this Dec 9, 2025

DouweM added the awaiting author revision label Dec 9, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add `tool_choice` setting #3611

Add `tool_choice` setting #3611

dsfaccini commented Dec 2, 2025 •

edited

Loading

Uh oh!

dsfaccini commented Dec 9, 2025

Uh oh!

DouweM Dec 9, 2025

Uh oh!

DouweM Dec 9, 2025

Uh oh!

DouweM Dec 9, 2025

Uh oh!

DouweM Dec 9, 2025

Uh oh!

DouweM Dec 9, 2025

Uh oh!

DouweM Dec 9, 2025

Uh oh!

DouweM Dec 9, 2025

Uh oh!

DouweM Dec 9, 2025

Uh oh!

DouweM Dec 9, 2025

Uh oh!

DouweM Dec 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		If both per-tool `prepare` and agent-wide `prepare_tools` are used, the per-tool `prepare` is applied first to each tool, and then `prepare_tools` is called with the resulting list of tool definitions.

		## Forcing Tool Use on First Request {#force-first-request}


		A common pattern is to force the model to use specific tools on the first request before allowing it to respond freely. This is particularly useful for RAG (Retrieval Augmented Generation) patterns where you want the model to search or retrieve information before answering.

		Use the `force_first_request` parameter on `@agent.tool` or `@agent.tool_plain` to force the model to use specific tools on the first model request of each [`run()`][pydantic_ai.Agent.run]:


		## Dynamic Tool Choice {#dynamic-tool-choice}

		For more complex scenarios, you can provide a callable for `tool_choice` in [`model_settings`][pydantic_ai.settings.ModelSettings] that dynamically determines which tools the model can use at each step:



		@dataclass
		class _OpenAIToolChoiceResult:

Add tool_choice setting #3611

Are you sure you want to change the base?

Add tool_choice setting #3611

Conversation

dsfaccini commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dsfaccini commented Dec 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add `tool_choice` setting #3611

Add `tool_choice` setting #3611

dsfaccini commented Dec 2, 2025 •

edited

Loading