-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Docs for PAIG #3382
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
laisbsc
wants to merge
14
commits into
main
Choose a base branch
from
docs/paig
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,567
−599
Draft
Docs for PAIG #3382
Changes from 3 commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
137de8a
add initial paig docs
laisbsc 006b739
add initial paig docs
laisbsc 76f51bb
Merge branch 'main' into docs/paig
laisbsc 0d3b92d
Merge branch 'main' into docs/paig
laisbsc 6d16271
add tab with `gateway` examples
laisbsc cec0f76
small edits on wording for docs
laisbsc 427dc4f
Update docs/ai_gateway/usage.md
laisbsc 9ea6712
Merge branch 'main' into docs/paig
laisbsc e9b9b31
Fix docs tests
DouweM 994583a
Set fake PAIG API key when testing examples
DouweM e46750a
Merge branch 'main' into docs/paig
laisbsc dacf5f4
fix tooltips on roulette_wheel.py example
laisbsc 850cab4
Reorganize gateway docs and update redirects
laisbsc ea2a720
Add Gateway tabs and fix tooltips in agents.md
laisbsc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # Quick Start | ||
| This page contains instructions on how to set up your account and run your app with AI Gateway credentials. | ||
|
|
||
| ## Create an account | ||
| Using your GitHub or Google account, sign in at https://gateway.pydantic.dev. | ||
| Choose a name for your organization (or accept the default). You will automatically be assigned the Admin role. | ||
|
|
||
| A default project will be created for you. You can choose to use it, or create a new one on the [Projects](https://gateway.pydantic.dev/admin/projects) page. | ||
|
|
||
| ## Add **Providers** by bringing your own API keys (BYOK) | ||
| Pydantic AI Gateway allows you to bring your API keys from your favourite provider(s). | ||
|
|
||
| On the [Providers](https://gateway.pydantic.dev/admin/providers) page, fill in the form to add a provider. Paste your API key into the form under Credentials, and make sure to **select the Project that will be associated to this provider**. It is possible to add multiple keys from the same provider. | ||
|
|
||
| ## Grant access to your team | ||
| On the [Users](https://gateway.pydantic.dev/admin/users) page, create an invitation and share the URL with your team to allow them to access the project. | ||
|
|
||
| ## Create gateway project keys | ||
| On the Keys page, Admins can create project keys which are not affected by spending limits. Users can only create personal keys, that will inherit spending caps from both User and Project levels, whichever is more restrictive. | ||
|
|
||
| [comment]: todo add screenshots or video |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| # Usage | ||
| After setting up your account with the instructions from the [Quick Start](./quick-start.md) page, you will be able to make an AI model request with the Pydantic AI Gateway. | ||
| This page contains the example code snippets you can use to test your keys with different frameworks and SDKs. | ||
|
|
||
| To use different models, change the model string `gateway/<api_type>:<model_name>` to other models offered by the supported providers. | ||
|
|
||
| Examples of providers and models that can be used are: | ||
|
|
||
| | **Provider** | **Provider ID** | **Example Model** | | ||
| | --- | --- | --- | | ||
| | OpenAI | `openai` | `gateway/openai:gpt-4.1` | | ||
| | Anthropic | `anthropic` | `gateway/anthropic:claude-sonnet-4-5` | | ||
| | Google Vertex | `google-vertex` | `gateway/google-vertex:gemini-2.5-flash` | | ||
| | Groq | `groq` | `gateway/groq:openai/gpt-oss-120b` | | ||
| | AWS Bedrock | `bedrock` | `gateway/bedrock:amazon.nova-micro-v1:0` | | ||
|
|
||
| ## Pydantic AI | ||
| Before you start, update to the latest version of `pydantic-ai`: | ||
|
|
||
| === "uv" | ||
|
|
||
| ```python | ||
| uv sync -P pydantic-ai | ||
| ``` | ||
| === "pip" | ||
|
|
||
| ```python | ||
| pip install -U pydantic-ai | ||
| ``` | ||
|
|
||
| Set the `PYDANTIC_AI_GATEWAY_API_KEY` environment variable to your gateway API key: | ||
|
|
||
| ```bash | ||
| export PYDANTIC_AI_GATEWAY_API_KEY="YOUR_PAIG_TOKEN" | ||
| ``` | ||
|
|
||
| You can access multiple models with the same API key, as shown in the code snippet below. | ||
|
|
||
| ```python title="hello_world.py" | ||
| from pydantic_ai import Agent | ||
|
|
||
| agent = Agent( | ||
| 'gateway/openai:gpt-4.1', | ||
| instructions='Be concise, reply with one sentence.' | ||
| ) | ||
|
|
||
| result = agent.run_sync('Hello World') | ||
| print(result.output) | ||
| ``` | ||
|
|
||
|
|
||
| ## Claude Code | ||
| Before you start, log out of Claude Code using `claude /logout`. | ||
|
|
||
| Set your gateway credentials as environment variables: | ||
|
|
||
| ```bash | ||
| export ANTHROPIC_AUTH_TOKEN="YOUR_PAIG_TOKEN" | ||
|
|
||
| export ANTHROPIC_BASE_URL="https://gateway.pydantic.dev/proxy/anthropic" | ||
| ``` | ||
|
|
||
| Replace `YOUR_PAIG_TOKEN` with the API key from the Keys page. | ||
|
|
||
| Launch Claude Code by typing `claude`. All requests will now route through the Pydantic AI Gateway. | ||
|
|
||
| ## SDKs | ||
|
|
||
| === "OpenAI SDK" | ||
|
|
||
| ```python title="hello_world.py" | ||
| import openai | ||
|
|
||
| client = openai.Client( | ||
| base_url='https://gateway.pydantic.dev/proxy/openai/', | ||
laisbsc marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| api_key='paig_...', | ||
| ) | ||
|
|
||
| response = client.chat.completions.create( | ||
| model='gpt-4o', | ||
| messages=[{'role': 'user', 'content': 'Hello world'}], | ||
| ) | ||
| print(response.choices[0].message.content) | ||
| ``` | ||
| === "Anthropic SDK" | ||
|
|
||
| ```python title="hello_world.py" | ||
| import anthropic | ||
|
|
||
| client = anthropic.Anthropic( | ||
| base_url='https://gateway.pydantic.dev/proxy/anthropic/', | ||
| auth_token='paig_...', | ||
| ) | ||
|
|
||
| response = client.messages.create( | ||
| max_tokens=1000, | ||
| model='claude-3-haiku-20240307', | ||
| messages=[{'role': 'user', 'content': 'Hello world'}], | ||
| ) | ||
| print(response.content[0].text) | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| # Pydantic AI Gateway | ||
|
|
||
| **Pydantic AI Gateway** (PAIG) is a unified interface for accessing multiple AI providers with a single key. Features include built-in OpenTelemetry observability, real-time cost monitoring, failover management , and native integration with the Pydantic stack. | ||
|
|
||
| !!! note "Currently in Beta" | ||
| The Pydantic AI Gateway is currently in Beta. No charges will be applied during this period. | ||
|
|
||
| Sign up at [gateway.pydantic.dev](https://gateway.pydantic.dev/). | ||
|
|
||
| For questions and feedback, contact us on [Slack](https://logfire.pydantic.dev/docs/join-slack/). | ||
|
|
||
| ## Key features | ||
| - **API key management**: access multiple LLM providers with a single API key. | ||
| - **Cost Limits**: set spending limits at project, user, and API key levels with daily, weekly, and monthly caps. | ||
| - **BYOK and managed providers:** Bring your own API keys (BYOK) from LLM providers, or pay for API usage directly through the platform (_coming soon_). | ||
| - **Multi-provider support:** Access models from OpenAI, Anthropic, Google Vertex, Groq, and AWS Bedrock. _More providers coming soon_. | ||
| - **Backend observability:** Log every request through [Pydantic Logfire](https://pydantic.dev/logfire) or any OpenTelemetry backend (_coming soon_). | ||
| - **Zero translation**: Unlike traditional AI gateways that translate everything to one common schema, PAIG allows requests to flow through directly in each provider's native format. This gives you immediate access to the new model features as soon as they are released. | ||
| - **Open source with self-hosting**: PAIG's core is [open source](https://github.com/pydantic/pydantic-ai-gateway/) (under [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.en.html)), allowing self-hosting with file-based configuration, instead of using the managed service. | ||
| - **Enterprise ready**: Includes SSO (with OIDC support), granular permissions, and flexible deployment options. Deploy to your Cloudflare account, or run on-premises with our [consulting support](https://pydantic.dev/contact). | ||
|
|
||
|
|
||
| ```python {title="hello_word.py"} | ||
| from pydantic_ai import Agent | ||
|
|
||
| agent = Agent( | ||
| 'gateway/openai:gpt-4.1', | ||
laisbsc marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| instructions='Be concise, reply with one sentence.' | ||
| ) | ||
|
|
||
| result = agent.run_sync('Hello World') | ||
| print(result.output) | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.