-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Update docs on multi-modal file URLs being downloaded or sent directly #3492
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
a1787d7
e3c404b
fd2d525
cac5057
16e5f56
0cbf87a
0249a4a
a09384b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -110,6 +110,10 @@ The situation is different for certain models: | |
|
|
||
| - [`AnthropicModel`][pydantic_ai.models.anthropic.AnthropicModel]: if you provide a PDF document via `DocumentUrl`, the URL is sent directly in the API request, so no download happens on the user side. | ||
|
|
||
| - [`GoogleModel`][pydantic_ai.models.google.GoogleModel] on GLA: | ||
| - YouTube video URLs are sent directly in the request to the model. | ||
| - Files uploaded to the [Files API](https://ai.google.dev/gemini-api/docs/files) can be passed by wrapping their url with DocumentUrl. | ||
|
|
||
| - [`GoogleModel`][pydantic_ai.models.google.GoogleModel] on Vertex AI: any URL provided using `ImageUrl`, `AudioUrl`, `VideoUrl`, or `DocumentUrl` is sent as-is in the API request and no data is downloaded beforehand. | ||
|
|
||
| See the [Gemini API docs for Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#filedata) to learn more about supported URLs, formats and limitations: | ||
|
|
@@ -120,4 +124,38 @@ The situation is different for certain models: | |
|
|
||
| However, because of crawling restrictions, it may happen that Gemini can't access certain URLs. In that case, you can instruct Pydantic AI to download the file content and send that instead of the URL by setting the boolean flag `force_download` to `True`. This attribute is available on all objects that inherit from [`FileUrl`][pydantic_ai.messages.FileUrl]. | ||
|
|
||
| - [`GoogleModel`][pydantic_ai.models.google.GoogleModel] on GLA: YouTube video URLs are sent directly in the request to the model. | ||
| ## Example code for Gemini GLA | ||
|
|
||
marmor7 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| Use uploaded files: | ||
|
|
||
| ```py {title="file_upload.py" test="skip" lint="skip"} | ||
| from pydantic_ai import Agent, DocumentUrl | ||
| from pydantic_ai.providers.google import GoogleProvider | ||
|
|
||
| provider = GoogleProvider(api_key=GEMINI_API_KEY) | ||
| file = provider.client.files.upload(file='path/to/document.pdf') | ||
| assert file.uri is not None | ||
|
|
||
| agent = Agent(model='google-gla:gemini-2.5-flash') | ||
| result = agent.run_sync( | ||
| [ | ||
| 'What does this document contain?', | ||
| DocumentUrl(url=file.uri, media_type=file.mime_type), | ||
| ] | ||
| ) | ||
| print(result.output) | ||
| ``` | ||
|
|
||
marmor7 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| Inline a file as a text part: | ||
|
|
||
| ```py {title="file_inline.py" test="skip" lint="skip"} | ||
| from pathlib import Path | ||
| from pydantic_ai import Agent, BinaryContent | ||
|
|
||
| file_bytes = Path('path/to/document.pdf').read_bytes() | ||
| mimetype = 'application/pdf' | ||
|
|
||
| agent = Agent(model='google-gla:gemini-2.5-flash') | ||
| result = agent.run_sync(['What does this document contain?', BinaryContent(data=file_bytes, media_type=mimetype)]) | ||
| print(result.output) | ||
| ``` | ||
Uh oh!
There was an error while loading. Please reload this page.