Skip to content

Commit 4376b96

Browse files
committed
✨ Add support for OpenAI and Gemini File Search Tools
- Add FileSearchTool builtin tool class - Implement OpenAI FileSearch tool support in OpenAIResponsesModel - Add _map_file_search_tool_call mapping function - Handle FileSearch in streaming and non-streaming responses - Add FileSearch to builtin tools list - Handle FileSearch in round-trip message conversion - Implement Gemini File Search tool support in GoogleModel - Add FileSearchTool handling in _get_tools method - Export FileSearchTool in __init__.py - Add comprehensive documentation in builtin-tools.md - Add tests for unsupported models This implements the feature requested in issue #3358. Fixes #3358
1 parent 889abfb commit 4376b96

File tree

6 files changed

+230
-10
lines changed

6 files changed

+230
-10
lines changed

docs/builtin-tools.md

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ Pydantic AI supports the following built-in tools:
1212
- **[`UrlContextTool`][pydantic_ai.builtin_tools.UrlContextTool]**: Enables agents to pull URL contents into their context
1313
- **[`MemoryTool`][pydantic_ai.builtin_tools.MemoryTool]**: Enables agents to use memory
1414
- **[`MCPServerTool`][pydantic_ai.builtin_tools.MCPServerTool]**: Enables agents to use remote MCP servers with communication handled by the model provider
15+
- **[`FileSearchTool`][pydantic_ai.builtin_tools.FileSearchTool]**: Enables agents to search through uploaded files using vector search (RAG)
1516

1617
These tools are passed to the agent via the `builtin_tools` parameter and are executed by the model provider's infrastructure.
1718

@@ -566,6 +567,99 @@ _(This example is complete, it can be run "as is")_
566567
| `description` |||
567568
| `headers` |||
568569

570+
## File Search Tool
571+
572+
The [`FileSearchTool`][pydantic_ai.builtin_tools.FileSearchTool] enables your agent to search through uploaded files using vector search, providing a fully managed Retrieval-Augmented Generation (RAG) system. This tool handles file storage, chunking, embedding generation, and context injection into prompts.
573+
574+
### Provider Support
575+
576+
| Provider | Supported | Notes |
577+
|----------|-----------|-------|
578+
| OpenAI Responses || Full feature support. Requires files to be uploaded to vector stores via the [OpenAI Files API](https://platform.openai.com/docs/api-reference/files). Vector stores must be created and file IDs added before using the tool. |
579+
| Google (Gemini) || Requires files to be uploaded via the [Gemini Files API](https://ai.google.dev/gemini-api/docs/files). Files are automatically deleted after 48 hours. Supports up to 2 GB per file and 20 GB per project. Using built-in tools and function tools (including [output tools](output.md#tool-output)) at the same time is not supported; to use structured output, use [`PromptedOutput`](output.md#prompted-output) instead. |
580+
| Anthropic || Not supported |
581+
| Groq || Not supported |
582+
| OpenAI Chat Completions || Not supported |
583+
| Bedrock || Not supported |
584+
| Mistral || Not supported |
585+
| Cohere || Not supported |
586+
| HuggingFace || Not supported |
587+
| Outlines || Not supported |
588+
589+
### Usage
590+
591+
#### OpenAI Responses
592+
593+
With OpenAI, you need to first upload files to a vector store, then reference the vector store IDs when using the `FileSearchTool`:
594+
595+
```py {title="file_search_openai.py"}
596+
from pydantic_ai import Agent, FileSearchTool
597+
598+
agent = Agent(
599+
'openai-responses:gpt-5',
600+
builtin_tools=[FileSearchTool(vector_store_ids=['vs_abc123'])] # (1)
601+
)
602+
603+
result = agent.run_sync('What information is in my documents about pydantic?')
604+
print(result.output)
605+
#> Based on your documents, Pydantic is a data validation library for Python...
606+
```
607+
608+
1. Replace `vs_abc123` with your actual vector store ID from the OpenAI API.
609+
610+
_(This example is complete, it can be run "as is")_
611+
612+
#### Google (Gemini)
613+
614+
With Gemini, you need to first upload files via the Files API, then reference the file resource names:
615+
616+
```py {title="file_search_google.py"}
617+
from pydantic_ai import Agent, FileSearchTool
618+
619+
agent = Agent(
620+
'google-gla:gemini-2.5-flash',
621+
builtin_tools=[FileSearchTool(vector_store_ids=['files/abc123'])] # (1)
622+
)
623+
624+
result = agent.run_sync('Summarize the key points from my uploaded documents.')
625+
print(result.output)
626+
#> The documents discuss the following key points: ...
627+
```
628+
629+
1. Replace `files/abc123` with your actual file resource name from the Gemini Files API.
630+
631+
_(This example is complete, it can be run "as is")_
632+
633+
!!! note "Gemini File Search API Status"
634+
The File Search Tool for Gemini was announced on November 6, 2025. The implementation may require adjustment as the official `google-genai` SDK is updated to fully support this feature.
635+
636+
### Configuration
637+
638+
The `FileSearchTool` accepts a list of vector store IDs:
639+
640+
- **OpenAI**: Vector store IDs created via the [OpenAI Files API](https://platform.openai.com/docs/api-reference/files)
641+
- **Google**: File resource names from the [Gemini Files API](https://ai.google.dev/gemini-api/docs/files)
642+
643+
```py {title="file_search_configured.py"}
644+
from pydantic_ai import Agent, FileSearchTool
645+
646+
agent = Agent(
647+
'openai-responses:gpt-5',
648+
builtin_tools=[
649+
FileSearchTool(
650+
vector_store_ids=['vs_store1', 'vs_store2'] # (1)
651+
)
652+
]
653+
)
654+
655+
result = agent.run_sync('Find information across all my document collections.')
656+
print(result.output)
657+
```
658+
659+
1. You can provide multiple vector store IDs to search across different collections.
660+
661+
_(This example is complete, it can be run "as is")_
662+
569663
## API Reference
570664

571665
For complete API documentation, see the [API Reference](api/builtin_tools.md).

pydantic_ai_slim/pydantic_ai/__init__.py

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
)
1212
from .builtin_tools import (
1313
CodeExecutionTool,
14+
FileSearchTool,
1415
ImageGenerationTool,
1516
MCPServerTool,
1617
MemoryTool,
@@ -210,13 +211,14 @@
210211
'ToolsetTool',
211212
'WrapperToolset',
212213
# builtin_tools
213-
'WebSearchTool',
214-
'WebSearchUserLocation',
215-
'UrlContextTool',
216214
'CodeExecutionTool',
215+
'FileSearchTool',
217216
'ImageGenerationTool',
218-
'MemoryTool',
219217
'MCPServerTool',
218+
'MemoryTool',
219+
'UrlContextTool',
220+
'WebSearchTool',
221+
'WebSearchUserLocation',
220222
# output
221223
'ToolOutput',
222224
'NativeOutput',

pydantic_ai_slim/pydantic_ai/builtin_tools.py

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
'ImageGenerationTool',
1818
'MemoryTool',
1919
'MCPServerTool',
20+
'FileSearchTool',
2021
)
2122

2223
_BUILTIN_TOOL_TYPES: dict[str, type[AbstractBuiltinTool]] = {}
@@ -334,6 +335,30 @@ def unique_id(self) -> str:
334335
return ':'.join([self.kind, self.id])
335336

336337

338+
@dataclass(kw_only=True)
339+
class FileSearchTool(AbstractBuiltinTool):
340+
"""A builtin tool that allows your agent to search through uploaded files using vector search.
341+
342+
This tool provides a fully managed Retrieval-Augmented Generation (RAG) system that handles
343+
file storage, chunking, embedding generation, and context injection into prompts.
344+
345+
Supported by:
346+
347+
* OpenAI Responses
348+
* Google (Gemini)
349+
"""
350+
351+
vector_store_ids: list[str]
352+
"""List of vector store IDs to search through.
353+
354+
For OpenAI, these are the IDs of vector stores created via the OpenAI API.
355+
For Google, these are file resource names that have been uploaded and processed.
356+
"""
357+
358+
kind: str = 'file_search'
359+
"""The kind of tool."""
360+
361+
337362
def _tool_discriminator(tool_data: dict[str, Any] | AbstractBuiltinTool) -> str:
338363
if isinstance(tool_data, dict):
339364
return tool_data.get('kind', AbstractBuiltinTool.kind)

pydantic_ai_slim/pydantic_ai/models/google.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
from .. import UnexpectedModelBehavior, _utils, usage
1414
from .._output import OutputObjectDefinition
1515
from .._run_context import RunContext
16-
from ..builtin_tools import CodeExecutionTool, ImageGenerationTool, UrlContextTool, WebSearchTool
16+
from ..builtin_tools import CodeExecutionTool, FileSearchTool, ImageGenerationTool, UrlContextTool, WebSearchTool
1717
from ..exceptions import UserError
1818
from ..messages import (
1919
BinaryContent,
@@ -342,6 +342,13 @@ def _get_tools(self, model_request_parameters: ModelRequestParameters) -> list[T
342342
tools.append(ToolDict(url_context=UrlContextDict()))
343343
elif isinstance(tool, CodeExecutionTool):
344344
tools.append(ToolDict(code_execution=ToolCodeExecutionDict()))
345+
elif isinstance(tool, FileSearchTool):
346+
# File Search Tool for Gemini API
347+
# The file_search tool uses file resource names (vector_store_ids) to search through uploaded files
348+
# Note: This requires files to be uploaded via the Files API first
349+
# The structure below is based on the Gemini File Search Tool announcement (Nov 2025)
350+
# and may require adjustment when the official google-genai SDK is updated
351+
tools.append(ToolDict(file_search={'file_names': tool.vector_store_ids})) # type: ignore[reportGeneralTypeIssues]
345352
elif isinstance(tool, ImageGenerationTool): # pragma: no branch
346353
if not self.profile.supports_image_output:
347354
raise UserError(

pydantic_ai_slim/pydantic_ai/models/openai.py

Lines changed: 75 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
from .._run_context import RunContext
1919
from .._thinking_part import split_content_into_text_and_thinking
2020
from .._utils import guard_tool_call_id as _guard_tool_call_id, now_utc as _now_utc, number_to_datetime
21-
from ..builtin_tools import CodeExecutionTool, ImageGenerationTool, MCPServerTool, WebSearchTool
21+
from ..builtin_tools import CodeExecutionTool, FileSearchTool, ImageGenerationTool, MCPServerTool, WebSearchTool
2222
from ..exceptions import UserError
2323
from ..messages import (
2424
AudioUrl,
@@ -1070,9 +1070,10 @@ def _process_response( # noqa: C901
10701070
elif isinstance(item, responses.response_output_item.LocalShellCall): # pragma: no cover
10711071
# Pydantic AI doesn't yet support the `codex-mini-latest` LocalShell built-in tool
10721072
pass
1073-
elif isinstance(item, responses.ResponseFileSearchToolCall): # pragma: no cover
1074-
# Pydantic AI doesn't yet support the FileSearch built-in tool
1075-
pass
1073+
elif isinstance(item, responses.ResponseFileSearchToolCall):
1074+
call_part, return_part = _map_file_search_tool_call(item, self.system)
1075+
items.append(call_part)
1076+
items.append(return_part)
10761077
elif isinstance(item, responses.response_output_item.McpCall):
10771078
call_part, return_part = _map_mcp_call(item, self.system)
10781079
items.append(call_part)
@@ -1267,6 +1268,11 @@ def _get_builtin_tools(self, model_request_parameters: ModelRequestParameters) -
12671268
type='approximate', **tool.user_location
12681269
)
12691270
tools.append(web_search_tool)
1271+
elif isinstance(tool, FileSearchTool):
1272+
file_search_tool = responses.FileSearchToolParam(
1273+
type='file_search', vector_store_ids=tool.vector_store_ids
1274+
)
1275+
tools.append(file_search_tool)
12701276
elif isinstance(tool, CodeExecutionTool):
12711277
has_image_generating_tool = True
12721278
tools.append({'type': 'code_interpreter', 'container': {'type': 'auto'}})
@@ -1404,6 +1410,7 @@ async def _map_messages( # noqa: C901
14041410
message_item: responses.ResponseOutputMessageParam | None = None
14051411
reasoning_item: responses.ResponseReasoningItemParam | None = None
14061412
web_search_item: responses.ResponseFunctionWebSearchParam | None = None
1413+
file_search_item: responses.ResponseFileSearchToolCallParam | None = None
14071414
code_interpreter_item: responses.ResponseCodeInterpreterToolCallParam | None = None
14081415
for item in message.parts:
14091416
if isinstance(item, TextPart):
@@ -1473,6 +1480,18 @@ async def _map_messages( # noqa: C901
14731480
type='web_search_call',
14741481
)
14751482
openai_messages.append(web_search_item)
1483+
elif (
1484+
item.tool_name == FileSearchTool.kind
1485+
and item.tool_call_id
1486+
and (args := item.args_as_dict())
1487+
):
1488+
file_search_item = responses.ResponseFileSearchToolCallParam(
1489+
id=item.tool_call_id,
1490+
action=cast(responses.response_file_search_tool_call_param.Action, args),
1491+
status='completed',
1492+
type='file_search_call',
1493+
)
1494+
openai_messages.append(file_search_item)
14761495
elif item.tool_name == ImageGenerationTool.kind and item.tool_call_id:
14771496
# The cast is necessary because of https://github.com/openai/openai-python/issues/2648
14781497
image_generation_item = cast(
@@ -1532,6 +1551,14 @@ async def _map_messages( # noqa: C901
15321551
and (status := content.get('status'))
15331552
):
15341553
web_search_item['status'] = status
1554+
elif (
1555+
item.tool_name == FileSearchTool.kind
1556+
and file_search_item is not None
1557+
and isinstance(item.content, dict) # pyright: ignore[reportUnknownMemberType]
1558+
and (content := cast(dict[str, Any], item.content)) # pyright: ignore[reportUnknownMemberType]
1559+
and (status := content.get('status'))
1560+
):
1561+
file_search_item['status'] = status
15351562
elif item.tool_name == ImageGenerationTool.kind:
15361563
# Image generation result does not need to be sent back, just the `id` off of `BuiltinToolCallPart`.
15371564
pass
@@ -1845,6 +1872,11 @@ async def _get_event_iterator(self) -> AsyncIterator[ModelResponseStreamEvent]:
18451872
yield self._parts_manager.handle_part(
18461873
vendor_part_id=f'{chunk.item.id}-call', part=replace(call_part, args=None)
18471874
)
1875+
elif isinstance(chunk.item, responses.ResponseFileSearchToolCall):
1876+
call_part, _ = _map_file_search_tool_call(chunk.item, self.provider_name)
1877+
yield self._parts_manager.handle_part(
1878+
vendor_part_id=f'{chunk.item.id}-call', part=replace(call_part, args=None)
1879+
)
18481880
elif isinstance(chunk.item, responses.ResponseCodeInterpreterToolCall):
18491881
call_part, _, _ = _map_code_interpreter_tool_call(chunk.item, self.provider_name)
18501882

@@ -1913,6 +1945,17 @@ async def _get_event_iterator(self) -> AsyncIterator[ModelResponseStreamEvent]:
19131945
elif isinstance(chunk.item, responses.ResponseFunctionWebSearch):
19141946
call_part, return_part = _map_web_search_tool_call(chunk.item, self.provider_name)
19151947

1948+
maybe_event = self._parts_manager.handle_tool_call_delta(
1949+
vendor_part_id=f'{chunk.item.id}-call',
1950+
args=call_part.args,
1951+
)
1952+
if maybe_event is not None: # pragma: no branch
1953+
yield maybe_event
1954+
1955+
yield self._parts_manager.handle_part(vendor_part_id=f'{chunk.item.id}-return', part=return_part)
1956+
elif isinstance(chunk.item, responses.ResponseFileSearchToolCall):
1957+
call_part, return_part = _map_file_search_tool_call(chunk.item, self.provider_name)
1958+
19161959
maybe_event = self._parts_manager.handle_tool_call_delta(
19171960
vendor_part_id=f'{chunk.item.id}-call',
19181961
args=call_part.args,
@@ -2216,6 +2259,34 @@ def _map_web_search_tool_call(
22162259
)
22172260

22182261

2262+
def _map_file_search_tool_call(
2263+
item: responses.ResponseFileSearchToolCall, provider_name: str
2264+
) -> tuple[BuiltinToolCallPart, BuiltinToolReturnPart]:
2265+
args: dict[str, Any] | None = None
2266+
2267+
result = {
2268+
'status': item.status,
2269+
}
2270+
2271+
if action := item.action:
2272+
args = action.model_dump(mode='json')
2273+
2274+
return (
2275+
BuiltinToolCallPart(
2276+
tool_name=FileSearchTool.kind,
2277+
tool_call_id=item.id,
2278+
args=args,
2279+
provider_name=provider_name,
2280+
),
2281+
BuiltinToolReturnPart(
2282+
tool_name=FileSearchTool.kind,
2283+
tool_call_id=item.id,
2284+
content=result,
2285+
provider_name=provider_name,
2286+
),
2287+
)
2288+
2289+
22192290
def _map_image_generation_tool_call(
22202291
item: responses.response_output_item.ImageGenerationCall, provider_name: str
22212292
) -> tuple[BuiltinToolCallPart, BuiltinToolReturnPart, FilePart | None]:

tests/test_builtin_tools.py

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
import pytest
44

55
from pydantic_ai.agent import Agent
6-
from pydantic_ai.builtin_tools import CodeExecutionTool, WebSearchTool
6+
from pydantic_ai.builtin_tools import CodeExecutionTool, FileSearchTool, WebSearchTool
77
from pydantic_ai.exceptions import UserError
88
from pydantic_ai.models import Model
99

@@ -40,3 +40,24 @@ async def test_builtin_tools_not_supported_code_execution_stream(model: Model, a
4040
with pytest.raises(UserError):
4141
async with agent.run_stream('What day is tomorrow?'):
4242
... # pragma: no cover
43+
44+
45+
@pytest.mark.parametrize(
46+
'model', ('bedrock', 'mistral', 'cohere', 'huggingface', 'groq', 'anthropic', 'test', 'outlines'), indirect=True
47+
)
48+
async def test_builtin_tools_not_supported_file_search(model: Model, allow_model_requests: None):
49+
agent = Agent(model=model, builtin_tools=[FileSearchTool(vector_store_ids=['test-id'])])
50+
51+
with pytest.raises(UserError):
52+
await agent.run('Search my files')
53+
54+
55+
@pytest.mark.parametrize(
56+
'model', ('bedrock', 'mistral', 'huggingface', 'groq', 'anthropic', 'outlines'), indirect=True
57+
)
58+
async def test_builtin_tools_not_supported_file_search_stream(model: Model, allow_model_requests: None):
59+
agent = Agent(model=model, builtin_tools=[FileSearchTool(vector_store_ids=['test-id'])])
60+
61+
with pytest.raises(UserError):
62+
async with agent.run_stream('Search my files'):
63+
... # pragma: no cover

0 commit comments

Comments
 (0)