Skip to content

Commit 8bbaa02

Browse files
laisbsclaisbsc
committed
consolidate docs in overview.md & update mkdocs
1 parent b8d2904 commit 8bbaa02

File tree

2 files changed

+164
-0
lines changed

2 files changed

+164
-0
lines changed

‎docs/ai_gateway/overview.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# Pydantic AI Gateway
2+
3+
**Pydantic AI Gateway** (PAIG) is a unified interface for accessing multiple AI providers with a single key. Features include built-in OpenTelemetry observability, real-time cost monitoring, failover management , and native integration with the Pydantic stack.
4+
5+
!!! note "Free while in Beta"
6+
The Pydantic AI Gateway is currently in Beta. You can bring your own key (BYOK) or buy inference through the Gateway (we will eat the card fee for now).
7+
8+
Sign up at [gateway.pydantic.dev](https://gateway.pydantic.dev/).
9+
10+
For questions and feedback, contact us on [Slack](https://logfire.pydantic.dev/docs/join-slack/).
11+
12+
## Documentation Integration
13+
14+
To help you get started with [Pydantic AI Gateway](https://gateway.pydantic.dev), most code examples throughout the Pydantic AI docs include a "Pydantic AI Gateway" tab alongside the standard "Pydantic AI" tab. This allows you to see how to adapt examples for Gateway usage by simply switching tabs.
15+
16+
The main difference is that when using Gateway, model strings use the `gateway/` prefix.
17+
18+
## Key features
19+
- **API key management**: access multiple LLM providers with a single Gateway key.
20+
- **Cost Limits**: set spending limits at project, user, and API key levels with daily, weekly, and monthly caps.
21+
- **BYOK and managed providers:** Bring your own API keys (BYOK) from LLM providers, or pay for API usage directly through the platform (_coming soon_).
22+
- **Multi-provider support:** Access models from OpenAI, Anthropic, Google Vertex, Groq, and AWS Bedrock. _More providers coming soon_.
23+
- **Backend observability:** Log every request through [Pydantic Logfire](https://pydantic.dev/logfire) or any OpenTelemetry backend (_coming soon_).
24+
- **Zero translation**: Unlike traditional AI gateways that translate everything to one common schema, PAIG allows requests to flow through directly in each provider's native format. This gives you immediate access to the new model features as soon as they are released.
25+
- **Open source with self-hosting**: PAIG's core is [open source](https://github.com/pydantic/pydantic-ai-gateway/) (under [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.en.html)), allowing self-hosting with file-based configuration, instead of using the managed service.
26+
- **Enterprise ready**: Includes SSO (with OIDC support), granular permissions, and flexible deployment options. Deploy to your Cloudflare account, or run on-premises with our [consulting support](https://pydantic.dev/contact).
27+
28+
```python {title="hello_word.py"}
29+
from pydantic_ai import Agent
30+
31+
agent = Agent('gateway/chat:gpt-4.1')
32+
33+
result = agent.run_sync('Where does "hello world" come from?')
34+
print(result.output)
35+
"""
36+
The first known use of "hello, world" was in a 1974 textbook about the C programming language.
37+
"""
38+
```
39+
# Quick Start
40+
This section contains instructions on how to set up your account and run your app with Pydantic AI Gateway credentials.
41+
42+
## Create an account
43+
Using your GitHub or Google account, sign in at https://gateway.pydantic.dev.
44+
Choose a name for your organization (or accept the default). You will automatically be assigned the Admin role.
45+
46+
A default project will be created for you. You can choose to use it, or create a new one on the [Projects](https://gateway.pydantic.dev/admin/projects) page.
47+
48+
## Add **Providers** by bringing your own API keys (BYOK)
49+
Pydantic AI Gateway allows you to bring your API keys from your favourite provider(s).
50+
51+
On the [Providers](https://gateway.pydantic.dev/admin/providers) page, fill in the form to add a provider. Paste your API key into the form under Credentials, and make sure to **select the Project that will be associated to this provider**. It is possible to add multiple keys from the same provider.
52+
53+
## Grant access to your team
54+
On the [Users](https://gateway.pydantic.dev/admin/users) page, create an invitation and share the URL with your team to allow them to access the project.
55+
56+
## Create gateway project keys
57+
On the Keys page, Admins can create project keys which are not affected by spending limits. Users can only create personal keys, that will inherit spending caps from both User and Project levels, whichever is more restrictive.
58+
59+
# Usage
60+
After setting up your account with the instructions above, you will be able to make an AI model request with the Pydantic AI Gateway.
61+
The code snippets below show how you can use PAIG with different frameworks and SDKs.
62+
63+
To use different models, change the model string `gateway/<api_type>:<model_name>` to other models offered by the supported providers.
64+
65+
Examples of providers and models that can be used are:
66+
67+
| **Provider** | **Provider ID** | **Example Model** |
68+
| --- | --- | --- |
69+
| OpenAI | `openai` | `gateway/openai:gpt-4.1` |
70+
| Anthropic | `anthropic` | `gateway/anthropic:claude-sonnet-4-5` |
71+
| Google Vertex | `google-vertex` | `gateway/google-vertex:gemini-2.5-flash` |
72+
| Groq | `groq` | `gateway/groq:openai/gpt-oss-120b` |
73+
| AWS Bedrock | `bedrock` | `gateway/bedrock:amazon.nova-micro-v1:0` |
74+
75+
## Pydantic AI
76+
Before you start, update to the latest version of `pydantic-ai`:
77+
78+
=== "uv"
79+
80+
```bash
81+
uv sync -P pydantic-ai
82+
```
83+
84+
=== "pip"
85+
86+
```bash
87+
pip install -U pydantic-ai
88+
```
89+
90+
Set the `PYDANTIC_AI_GATEWAY_API_KEY` environment variable to your gateway API key:
91+
92+
```bash
93+
export PYDANTIC_AI_GATEWAY_API_KEY="YOUR_PAIG_TOKEN"
94+
```
95+
96+
You can access multiple models with the same API key, as shown in the code snippet below.
97+
98+
```python title="hello_world.py"
99+
from pydantic_ai import Agent
100+
101+
agent = Agent('gateway/chat:gpt-5')
102+
103+
result = agent.run_sync('Where does "hello world" come from?')
104+
print(result.output)
105+
"""
106+
The first known use of "hello, world" was in a 1974 textbook about the C programming language.
107+
"""
108+
```
109+
110+
111+
## Claude Code
112+
Before you start, log out of Claude Code using `/logout`.
113+
114+
Set your gateway credentials as environment variables:
115+
116+
```bash
117+
export ANTHROPIC_BASE_URL="https://gateway.pydantic.dev/proxy/anthropic"
118+
export ANTHROPIC_AUTH_TOKEN="YOUR_PAIG_TOKEN"
119+
```
120+
121+
Replace `YOUR_PAIG_TOKEN` with the API key from the Keys page.
122+
123+
Launch Claude Code by typing `claude`. All requests will now route through the Pydantic AI Gateway.
124+
125+
## SDKs
126+
127+
=== "OpenAI SDK"
128+
129+
```python {title="openai_sdk.py" test="skip"}
130+
import openai
131+
132+
client = openai.Client(
133+
base_url='https://gateway.pydantic.dev/proxy/chat/',
134+
api_key='paig_...',
135+
)
136+
137+
response = client.chat.completions.create(
138+
model='gpt-4o',
139+
messages=[{'role': 'user', 'content': 'Hello world'}],
140+
)
141+
print(response.choices[0].message.content)
142+
#> Hello user
143+
```
144+
=== "Anthropic SDK"
145+
146+
```python {title="anthropic_sdk.py" test="skip"}
147+
import anthropic
148+
149+
client = anthropic.Anthropic(
150+
base_url='https://gateway.pydantic.dev/proxy/anthropic/',
151+
auth_token='paig_...',
152+
)
153+
154+
response = client.messages.create(
155+
max_tokens=1000,
156+
model='claude-3-haiku-20240307',
157+
messages=[{'role': 'user', 'content': 'Hello world'}],
158+
)
159+
print(response.content[0].text)
160+
#> Hello user
161+
```

‎mkdocs.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,9 @@ nav:
5454
- Multi-Agent Patterns: multi-agent-applications.md
5555
- Testing: testing.md
5656

57+
- Pydantic AI Gateway:
58+
- Overview: ai_gateway/overview.md
59+
5760
- Pydantic Evals:
5861
- Overview: evals.md
5962
- Getting Started:

0 commit comments

Comments
 (0)