Skip to content

Commit 07f54f9

Browse files
Document performance implications of async vs sync tools (#2298)
Co-authored-by: Douwe Maan <[email protected]>
1 parent 74d94ec commit 07f54f9

File tree

1 file changed

+6
-0
lines changed

1 file changed

+6
-0
lines changed

docs/tools.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -724,6 +724,12 @@ def my_flaky_tool(query: str) -> str:
724724

725725
Raising `ModelRetry` also generates a `RetryPromptPart` containing the exception message, which is sent back to the LLM to guide its next attempt. Both `ValidationError` and `ModelRetry` respect the `retries` setting configured on the `Tool` or `Agent`.
726726

727+
### Parallel tool calls & concurrency
728+
729+
When a model returns multiple tool calls in one response, Pydantic AI schedules them concurrently using `asyncio.create_task`.
730+
731+
Async functions are run on the event loop, while sync functions are offloaded to threads. To get the best performance, _always_ use an async function _unless_ you're doing blocking I/O (and there's no way to use a non-blocking library instead) or CPU-bound work (like `numpy` or `scikit-learn` operations), so that simple functions are not offloaded to threads unnecessarily.
732+
727733
## Third-Party Tools
728734

729735
### MCP Tools {#mcp-tools}

0 commit comments

Comments
 (0)