Skip to content

Commit 5216c6e

Browse files
dmontagudmontagu
committed
Reorganize the tools docs
1 parent 66fa21a commit 5216c6e

File tree

5 files changed

+831
-786
lines changed

5 files changed

+831
-786
lines changed

‎docs/deferred-tools.md‎

Lines changed: 324 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,324 @@
1+
# Deferred Tools
2+
3+
There are a few scenarios where the model should be able to call a tool that should not or cannot be executed during the same agent run inside the same Python process:
4+
5+
- it may need to be approved by the user first
6+
- it may depend on an upstream service, frontend, or user to provide the result
7+
- the result could take longer to generate than it's reasonable to keep the agent process running
8+
9+
To support these use cases, Pydantic AI provides the concept of deferred tools, which come in two flavors documented below:
10+
11+
- tools that [require approval](#human-in-the-loop-tool-approval)
12+
- tools that are [executed externally](#external-tool-execution)
13+
14+
When the model calls a deferred tool, the agent run will end with a [`DeferredToolRequests`][pydantic_ai.output.DeferredToolRequests] output object containing information about the deferred tool calls. Once the approvals and/or results are ready, a new agent run can then be started with the original run's [message history](message-history.md) plus a [`DeferredToolResults`][pydantic_ai.tools.DeferredToolResults] object holding results for each tool call in `DeferredToolRequests`, which will continue the original run where it left off.
15+
16+
Note that handling deferred tool calls requires `DeferredToolRequests` to be in the `Agent`'s [`output_type`](output.md#structured-output) so that the possible types of the agent run output are correctly inferred. If your agent can also be used in a context where no deferred tools are available and you don't want to deal with that type everywhere you use the agent, you can instead pass the `output_type` argument when you run the agent using [`agent.run()`][pydantic_ai.agent.AbstractAgent.run], [`agent.run_sync()`][pydantic_ai.agent.AbstractAgent.run_sync], [`agent.run_stream()`][pydantic_ai.agent.AbstractAgent.run_stream], or [`agent.iter()`][pydantic_ai.Agent.iter]. Note that the run-time `output_type` overrides the one specified at construction time (for type inference reasons), so you'll need to include the original output type explicitly.
17+
18+
## Human-in-the-Loop Tool Approval
19+
20+
If a tool function always requires approval, you can pass the `requires_approval=True` argument to the [`@agent.tool`][pydantic_ai.Agent.tool] decorator, [`@agent.tool_plain`][pydantic_ai.Agent.tool_plain] decorator, [`Tool`][pydantic_ai.tools.Tool] class, [`FunctionToolset.tool`][pydantic_ai.toolsets.FunctionToolset.tool] decorator, or [`FunctionToolset.add_function()`][pydantic_ai.toolsets.FunctionToolset.add_function] method. Inside the function, you can then assume that the tool call has been approved.
21+
22+
If whether a tool function requires approval depends on the tool call arguments or the agent [run context][pydantic_ai.tools.RunContext] (e.g. [dependencies](dependencies.md) or message history), you can raise the [`ApprovalRequired`][pydantic_ai.exceptions.ApprovalRequired] exception from the tool function. The [`RunContext.tool_call_approved`][pydantic_ai.tools.RunContext.tool_call_approved] property will be `True` if the tool call has already been approved.
23+
24+
To require approval for calls to tools provided by a [toolset](toolsets.md) (like an [MCP server](mcp/client.md)), see the [`ApprovalRequiredToolset` documentation](toolsets.md#requiring-tool-approval).
25+
26+
When the model calls a tool that requires approval, the agent run will end with a [`DeferredToolRequests`][pydantic_ai.output.DeferredToolRequests] output object with an `approvals` list holding [`ToolCallPart`s][pydantic_ai.messages.ToolCallPart] containing the tool name, validated arguments, and a unique tool call ID.
27+
28+
Once you've gathered the user's approvals or denials, you can build a [`DeferredToolResults`][pydantic_ai.tools.DeferredToolResults] object with an `approvals` dictionary that maps each tool call ID to a boolean, a [`ToolApproved`][pydantic_ai.tools.ToolApproved] object (with optional `override_args`), or a [`ToolDenied`][pydantic_ai.tools.ToolDenied] object (with an optional custom `message` to provide to the model). This `DeferredToolResults` object can then be provided to one of the agent run methods as `deferred_tool_results`, alongside the original run's [message history](message-history.md).
29+
30+
Here's an example that shows how to require approval for all file deletions, and for updates of specific protected files:
31+
32+
```python {title="tool_requires_approval.py"}
33+
from pydantic_ai import (
34+
Agent,
35+
ApprovalRequired,
36+
DeferredToolRequests,
37+
DeferredToolResults,
38+
RunContext,
39+
ToolDenied,
40+
)
41+
42+
agent = Agent('openai:gpt-5', output_type=[str, DeferredToolRequests])
43+
44+
PROTECTED_FILES = {'.env'}
45+
46+
47+
@agent.tool
48+
def update_file(ctx: RunContext, path: str, content: str) -> str:
49+
if path in PROTECTED_FILES and not ctx.tool_call_approved:
50+
raise ApprovalRequired
51+
return f'File {path!r} updated: {content!r}'
52+
53+
54+
@agent.tool_plain(requires_approval=True)
55+
def delete_file(path: str) -> str:
56+
return f'File {path!r} deleted'
57+
58+
59+
result = agent.run_sync('Delete `__init__.py`, write `Hello, world!` to `README.md`, and clear `.env`')
60+
messages = result.all_messages()
61+
62+
assert isinstance(result.output, DeferredToolRequests)
63+
requests = result.output
64+
print(requests)
65+
"""
66+
DeferredToolRequests(
67+
calls=[],
68+
approvals=[
69+
ToolCallPart(
70+
tool_name='update_file',
71+
args={'path': '.env', 'content': ''},
72+
tool_call_id='update_file_dotenv',
73+
),
74+
ToolCallPart(
75+
tool_name='delete_file',
76+
args={'path': '__init__.py'},
77+
tool_call_id='delete_file',
78+
),
79+
],
80+
)
81+
"""
82+
83+
results = DeferredToolResults()
84+
for call in requests.approvals:
85+
result = False
86+
if call.tool_name == 'update_file':
87+
# Approve all updates
88+
result = True
89+
elif call.tool_name == 'delete_file':
90+
# deny all deletes
91+
result = ToolDenied('Deleting files is not allowed')
92+
93+
results.approvals[call.tool_call_id] = result
94+
95+
result = agent.run_sync(message_history=messages, deferred_tool_results=results)
96+
print(result.output)
97+
"""
98+
I successfully updated `README.md` and cleared `.env`, but was not able to delete `__init__.py`.
99+
"""
100+
print(result.all_messages())
101+
"""
102+
[
103+
ModelRequest(
104+
parts=[
105+
UserPromptPart(
106+
content='Delete `__init__.py`, write `Hello, world!` to `README.md`, and clear `.env`',
107+
timestamp=datetime.datetime(...),
108+
)
109+
]
110+
),
111+
ModelResponse(
112+
parts=[
113+
ToolCallPart(
114+
tool_name='delete_file',
115+
args={'path': '__init__.py'},
116+
tool_call_id='delete_file',
117+
),
118+
ToolCallPart(
119+
tool_name='update_file',
120+
args={'path': 'README.md', 'content': 'Hello, world!'},
121+
tool_call_id='update_file_readme',
122+
),
123+
ToolCallPart(
124+
tool_name='update_file',
125+
args={'path': '.env', 'content': ''},
126+
tool_call_id='update_file_dotenv',
127+
),
128+
],
129+
usage=RequestUsage(input_tokens=63, output_tokens=21),
130+
model_name='gpt-5',
131+
timestamp=datetime.datetime(...),
132+
),
133+
ModelRequest(
134+
parts=[
135+
ToolReturnPart(
136+
tool_name='delete_file',
137+
content='Deleting files is not allowed',
138+
tool_call_id='delete_file',
139+
timestamp=datetime.datetime(...),
140+
),
141+
ToolReturnPart(
142+
tool_name='update_file',
143+
content="File 'README.md' updated: 'Hello, world!'",
144+
tool_call_id='update_file_readme',
145+
timestamp=datetime.datetime(...),
146+
),
147+
ToolReturnPart(
148+
tool_name='update_file',
149+
content="File '.env' updated: ''",
150+
tool_call_id='update_file_dotenv',
151+
timestamp=datetime.datetime(...),
152+
),
153+
]
154+
),
155+
ModelResponse(
156+
parts=[
157+
TextPart(
158+
content='I successfully updated `README.md` and cleared `.env`, but was not able to delete `__init__.py`.'
159+
)
160+
],
161+
usage=RequestUsage(input_tokens=79, output_tokens=39),
162+
model_name='gpt-5',
163+
timestamp=datetime.datetime(...),
164+
),
165+
]
166+
"""
167+
```
168+
169+
_(This example is complete, it can be run "as is")_
170+
171+
## External Tool Execution
172+
173+
When the result of a tool call cannot be generated inside the same agent run in which it was called, the tool is considered to be external.
174+
Examples of external tools are client-side tools implemented by a web or app frontend, and slow tasks that are passed off to a background worker or external service instead of keeping the agent process running.
175+
176+
If whether a tool call should be executed externally depends on the tool call arguments, the agent [run context][pydantic_ai.tools.RunContext] (e.g. [dependencies](dependencies.md) or message history), or how long the task is expected to take, you can define a tool function and conditionally raise the [`CallDeferred`][pydantic_ai.exceptions.CallDeferred] exception. Before raising the exception, the tool function would typically schedule some background task and pass along the [`RunContext.tool_call_id`][pydantic_ai.tools.RunContext.tool_call_id] so that the result can be matched to the deferred tool call later.
177+
178+
If a tool is always executed externally and its definition is provided to your code along with a JSON schema for its arguments, you can use an [`ExternalToolset`](toolsets.md#external-toolset). If the external tools are known up front and you don't have the arguments JSON schema handy, you can also define a tool function with the appropriate signature that does nothing but raise the [`CallDeferred`][pydantic_ai.exceptions.CallDeferred] exception.
179+
180+
When the model calls an external tool, the agent run will end with a [`DeferredToolRequests`][pydantic_ai.output.DeferredToolRequests] output object with a `calls` list holding [`ToolCallPart`s][pydantic_ai.messages.ToolCallPart] containing the tool name, validated arguments, and a unique tool call ID.
181+
182+
Once the tool call results are ready, you can build a [`DeferredToolResults`][pydantic_ai.tools.DeferredToolResults] object with a `calls` dictionary that maps each tool call ID to an arbitrary value to be returned to the model, a [`ToolReturn`](tools-advanced.md#advanced-tool-returns) object, or a [`ModelRetry`][pydantic_ai.exceptions.ModelRetry] exception in case the tool call failed and the model should [try again](tools-advanced.md#tool-retries). This `DeferredToolResults` object can then be provided to one of the agent run methods as `deferred_tool_results`, alongside the original run's [message history](message-history.md).
183+
184+
Here's an example that shows how to move a task that takes a while to complete to the background and return the result to the model once the task is complete:
185+
186+
```python {title="external_tool.py"}
187+
import asyncio
188+
from dataclasses import dataclass
189+
from typing import Any
190+
191+
from pydantic_ai import (
192+
Agent,
193+
CallDeferred,
194+
DeferredToolRequests,
195+
DeferredToolResults,
196+
ModelRetry,
197+
RunContext,
198+
)
199+
200+
201+
@dataclass
202+
class TaskResult:
203+
tool_call_id: str
204+
result: Any
205+
206+
207+
async def calculate_answer_task(tool_call_id: str, question: str) -> TaskResult:
208+
await asyncio.sleep(1)
209+
return TaskResult(tool_call_id=tool_call_id, result=42)
210+
211+
212+
agent = Agent('openai:gpt-5', output_type=[str, DeferredToolRequests])
213+
214+
tasks: list[asyncio.Task[TaskResult]] = []
215+
216+
217+
@agent.tool
218+
async def calculate_answer(ctx: RunContext, question: str) -> str:
219+
assert ctx.tool_call_id is not None
220+
221+
task = asyncio.create_task(calculate_answer_task(ctx.tool_call_id, question)) # (1)!
222+
tasks.append(task)
223+
224+
raise CallDeferred
225+
226+
227+
async def main():
228+
result = await agent.run('Calculate the answer to the ultimate question of life, the universe, and everything')
229+
messages = result.all_messages()
230+
231+
assert isinstance(result.output, DeferredToolRequests)
232+
requests = result.output
233+
print(requests)
234+
"""
235+
DeferredToolRequests(
236+
calls=[
237+
ToolCallPart(
238+
tool_name='calculate_answer',
239+
args={
240+
'question': 'the ultimate question of life, the universe, and everything'
241+
},
242+
tool_call_id='pyd_ai_tool_call_id',
243+
)
244+
],
245+
approvals=[],
246+
)
247+
"""
248+
249+
done, _ = await asyncio.wait(tasks) # (2)!
250+
task_results = [task.result() for task in done]
251+
task_results_by_tool_call_id = {result.tool_call_id: result.result for result in task_results}
252+
253+
results = DeferredToolResults()
254+
for call in requests.calls:
255+
try:
256+
result = task_results_by_tool_call_id[call.tool_call_id]
257+
except KeyError:
258+
result = ModelRetry('No result for this tool call was found.')
259+
260+
results.calls[call.tool_call_id] = result
261+
262+
result = await agent.run(message_history=messages, deferred_tool_results=results)
263+
print(result.output)
264+
#> The answer to the ultimate question of life, the universe, and everything is 42.
265+
print(result.all_messages())
266+
"""
267+
[
268+
ModelRequest(
269+
parts=[
270+
UserPromptPart(
271+
content='Calculate the answer to the ultimate question of life, the universe, and everything',
272+
timestamp=datetime.datetime(...),
273+
)
274+
]
275+
),
276+
ModelResponse(
277+
parts=[
278+
ToolCallPart(
279+
tool_name='calculate_answer',
280+
args={
281+
'question': 'the ultimate question of life, the universe, and everything'
282+
},
283+
tool_call_id='pyd_ai_tool_call_id',
284+
)
285+
],
286+
usage=RequestUsage(input_tokens=63, output_tokens=13),
287+
model_name='gpt-5',
288+
timestamp=datetime.datetime(...),
289+
),
290+
ModelRequest(
291+
parts=[
292+
ToolReturnPart(
293+
tool_name='calculate_answer',
294+
content=42,
295+
tool_call_id='pyd_ai_tool_call_id',
296+
timestamp=datetime.datetime(...),
297+
)
298+
]
299+
),
300+
ModelResponse(
301+
parts=[
302+
TextPart(
303+
content='The answer to the ultimate question of life, the universe, and everything is 42.'
304+
)
305+
],
306+
usage=RequestUsage(input_tokens=64, output_tokens=28),
307+
model_name='gpt-5',
308+
timestamp=datetime.datetime(...),
309+
),
310+
]
311+
"""
312+
```
313+
314+
1. In reality, you'd likely use Celery or a similar task queue to run the task in the background.
315+
2. In reality, this would typically happen in a separate process that polls for the task status or is notified when all pending tasks are complete.
316+
317+
_(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_
318+
319+
## See Also
320+
321+
- [Function Tools](tools.md) - Basic tool concepts and registration
322+
- [Advanced Tool Features](tools-advanced.md) - Custom schemas, dynamic tools, and execution details
323+
- [Toolsets](toolsets.md) - Managing collections of tools, including `ExternalToolset` for external tools
324+
- [Message History](message-history.md) - Understanding how to work with message history for deferred tools

0 commit comments

Comments
 (0)