Skip to content
Open
63 changes: 54 additions & 9 deletions docs/gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,18 +119,63 @@ export PYDANTIC_AI_GATEWAY_API_KEY="YOUR_PYDANTIC_AI_GATEWAY_API_KEY"

You can access multiple models with the same API key, as shown in the code snippet below.

```python {title="hello_world.py"}
from pydantic_ai import Agent
=== "Hello World"

agent = Agent('gateway/openai:gpt-5')
```python {title="hello_world.py"}
from pydantic_ai import Agent

result = agent.run_sync('Where does "hello world" come from?')
print(result.output)
"""
The first known use of "hello, world" was in a 1974 textbook about the C programming language.
"""
```
agent = Agent('gateway/openai:gpt-5')

result = agent.run_sync('Where does "hello world" come from?')
print(result.output)
"""
The first known use of "hello, world" was in a 1974 textbook about the C programming language.
"""
```

=== "Passing API Key directly"

Pass your API key directly using the provider:

```python {title="passing_api_key.py"}
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.gateway import gateway_provider
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should link to the API docs for this function, so people can read the docstrings etc


provider = gateway_provider('openai', api_key='PYDANTIC_AI_GATEWAY_API_KEY')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor, but I feel like it makes sense to make it more clear that this isn't the literal string you are supposed to enter

Suggested change
provider = gateway_provider('openai', api_key='PYDANTIC_AI_GATEWAY_API_KEY')
provider = gateway_provider('openai', api_key='paig_<example_key>')

or similar. (If it needs to match elsewhere, we can change it elsewhere?)

The current value from this example is the exact name of the environment variable that is used, which imo has just a little bit of potential for confusion

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might make sense to make use of the route parameter too as that's a particularly relevant one; if you do, I'd use route='builtin-openai' or similar — those should always work.

(route='builtin-openai' means to use specifically the builtin openai provider, even if you've added your own OpenAI provider to the builtin routing group (which will have fallback.).)

I'm okay leaving that out for now, up to you.

model = OpenAIChatModel('gpt-5', provider=provider)
agent = Agent(model)

result = agent.run_sync('Where does "hello world" come from?')
print(result.output)
"""
The first known use of "hello, world" was in a 1974 textbook about the C programming language.
"""
```

=== "Using a different upstream provider"

If you're using a different upstream provider, you can specify it in the route parameter:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmontagu Can you review the terminology here please? I don't think route and upstream provider are interchangeable like this


```python {title="routing_via_provider.py"}
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.gateway import gateway_provider

provider = gateway_provider(
'openai',
api_key='PYDANTIC_AI_GATEWAY_API_KEY',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same suggestion here

route='modal-ai'
)
model = OpenAIChatModel('gpt-5', provider=provider)
agent = Agent(model)

result = agent.run_sync('Where does "hello world" come from?')
print(result.output)
"""
The first known use of "hello, world" was in a 1974 textbook about the C programming language.
"""
```

### Claude Code

Expand Down