-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Initial Checks
- I confirm that I'm using the latest version of Pydantic AI
- I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue
Description
I'm specifically trying to use Claude Sonnet 4 on VertexAI as my main model, with fallback on the Anthropic API with 1m context window as a second model fallback. When exceeding the VertexAI 200k token window, the request fails entirely. The issue persists if I remove the VertexAI model and have only the Anthropic API model wrapped in FallbackModel
If I unwrap the Anthropic API model definition (as shown below) from being inside FallbackModel it works going beyond the 200k token window.
Example Code
from pydantic_ai import Agent
from pydantic_ai.models.anthropic import AnthropicModel, AnthropicModelSettings
from pydantic_ai.providers.anthropic import AnthropicProvider
claude_sonnet_4_model = AnthropicModel(
"claude-sonnet-4-20250514",
provider=AnthropicProvider(
anthropic_client=anthropic_client,
),
settings=AnthropicModelSettings(
extra_headers={
"anthropic-beta": "context-1m-2025-08-07"
}
)
)
agent = Agent(
model=claude_sonnet_4_model,
# If the model is wrapped in a FallbackModel, the `anthropic-beta` header isn't used
# model=FallbackModel(claude_sonnet_4_model),
model_settings=AnthropicModelSettings(
temperature=1.0, max_tokens=1024
)
)Python, Pydantic AI & LLM client version
pydantic-ai 0.6.2
Python 3.12
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working