Add WebFetchTool builtin tool support #3427
Conversation
|
@DouweM would you prefer if we renamed UrlContextTool to WebFetchTool and left UrlContextTool in a google-only deprecated state? |
docs/builtin-tools.md
Outdated
|
|
||
| _(This example is complete, it can be run "as is")_ | ||
|
|
||
| With Google, you can also use `UrlContextTool`: |
There was a problem hiding this comment.
We just need one example as the only difference is the model name.
Per the above, let's not mention UrlContextTool anymore.
Should we support any of the options on https://docs.claude.com/en/docs/agents-and-tools/tool-use/web-fetch-tool#tool-definition? If so, that'd warrant a new section and Anthropic-specific example. But ideally Google would also support at least some of those.
There was a problem hiding this comment.
Good idea! I'll look into adding those options
tests/models/test_anthropic.py
Outdated
| assert len(tool_calls) >= 1 | ||
| assert len(tool_returns) >= 1 | ||
| assert any(tc.tool_name == 'url_context' for tc in tool_calls) | ||
| assert any(tr.tool_name == 'url_context' for tr in tool_returns) |
There was a problem hiding this comment.
Please use full snapshots like in the other builtin tool tests
tests/models/test_anthropic.py
Outdated
| 'Pydantic AI is a Python agent framework designed to help you quickly, confidently, and painlessly build production grade applications and workflows with Generative AI.' | ||
| ) | ||
|
|
||
| messages = agent_run.result.all_messages() |
There was a problem hiding this comment.
Same as above; full snapshots of messages and events please
tests/models/test_anthropic.py
Outdated
|
|
||
|
|
||
| @pytest.mark.vcr() | ||
| async def test_anthropic_url_context_tool_multi_turn(allow_model_requests: None, anthropic_api_key: str): |
There was a problem hiding this comment.
See tests for other builtin tools: we typically check this by having 2 agent.runs in the same non-streaming test, where the second takes the history of the first to ensure that the API accepts it.
tests/models/test_anthropic.py
Outdated
| assert len(anthropic_messages) == 0 # No messages should be added | ||
|
|
||
|
|
||
| @pytest.mark.vcr() |
There was a problem hiding this comment.
Why were these changes necessary?
tests/models/test_anthropic.py
Outdated
| ) | ||
|
|
||
| result = await agent.run('How much is 3 * 12390?') | ||
| result = await agent.run('How much is 3 * 12390?') # pragma: lax no cover |
There was a problem hiding this comment.
Please remove all the new # pragma: lax no covers, I don't think they should be needed
sarth6
changed the title
|
@DouweM I ended up adding the Anthropic WebFetch params to our new builtin tool, but looks like Google's URL Context Tool doesn't support any of them as far as I could see from their docs, as the UrlContextToolDict schema is empty https://ai.google.dev/gemini-api/docs/url-context#contextual-response |
| | Provider | Supported | Notes | | ||
| |----------|-----------|-------| | ||
| | Anthropic | ✅ | Full feature support. Uses Anthropic's [Web Fetch Tool](https://docs.claude.com/en/docs/agents-and-tools/tool-use/web-fetch-tool) internally to retrieve URL contents. | | ||
| | Google | ✅ | No [`BuiltinToolCallPart`][pydantic_ai.messages.BuiltinToolCallPart] or [`BuiltinToolReturnPart`][pydantic_ai.messages.BuiltinToolReturnPart] is currently generated; please submit an issue if you need this. Using built-in tools and function tools (including [output tools](output.md#tool-output)) at the same time is not supported; to use structured output, use [`PromptedOutput`](output.md#prompted-output) instead. | |
There was a problem hiding this comment.
While we're at it, would you be up for fixing "No [BuiltinToolCallPart][pydantic_ai.messages.BuiltinToolCallPart] or [BuiltinToolReturnPart][pydantic_ai.messages.BuiltinToolReturnPart] is currently generated"?
Per https://ai.google.dev/gemini-api/docs/url-context#contextual-response the data is available, and I see the same in the test_google_model_url_context_tool cassette, so you should be able to get it to transform into parts nicely without even needing to regenerate the cassette (although I suppose we'd ideally want a streaming test + cassette as well).
It may be better to do that in a future PR (not necessarily you), unless you feel like doing it now :)
|
|
||
| _(This example is complete, it can be run "as is")_ | ||
|
|
||
| ### Parameters |
There was a problem hiding this comment.
We call them Configuration Options in all the other examples; please make sure the wording is consistent, as well as the way the table is structured, the Provider Support subsection, etc.
| With Anthropic, you can only use one of `blocked_domains` or `allowed_domains`, not both. | ||
|
|
||
| !!! note | ||
| Google's URL context tool does not support any configuration parameters. The limits are fixed at 20 URLs per request with a maximum of 34MB per URL. |
There was a problem hiding this comment.
This should be in a Provider Support Notes column
|
|
||
|
|
||
| # Remove UrlContextTool from _BUILTIN_TOOL_TYPES and restore WebFetchTool | ||
| # This ensures the discriminated union only includes WebFetchTool |
There was a problem hiding this comment.
Would that cause issues with old payloads that are being deserialized now? Or old code that is now giving a deprecation warning but hasn't actually been updated yet? Would be worth testing in test_builtin_tools.
| tools.append(BetaCodeExecutionTool20250522Param(name='code_execution', type='code_execution_20250522')) | ||
| beta_features.append('code-execution-2025-05-22') | ||
| elif isinstance(tool, WebFetchTool): # pragma: no branch | ||
| citations = BetaCitationsConfigParam(enabled=tool.citations_enabled) if tool.citations_enabled else None |
There was a problem hiding this comment.
Let's name our field enable_citations,
2 participants
Closes #2971
WebFetchToolbuiltin tool for Anthropic/Google models, which uses the (Claude WebFetch tool / Google URL Context tool) under-the-hoodUrlContextTool(which was Google-only) as users should now useWebFetchToolinsteadBetaWebFetchToolResultBlockParamContent(web fetch url / retrieved_at / source data / etc) in the Pydantic AIBuiltinToolReturnPartthat arrives in the agent message history, so that Pydantic AI users have access to the full web fetch tool return objectWebFetchToolthat get passed to the Claude WebFetch tool under-the-hood; these params are not supported by Google'sUrlContextToolDictunfortunately