Skip to content

ModelSettings merging not working when wrapped with FallbackModel #2539

@jerry-reevo

Description

@jerry-reevo

Initial Checks

Description

I'm specifically trying to use Claude Sonnet 4 on VertexAI as my main model, with fallback on the Anthropic API with 1m context window as a second model fallback. When exceeding the VertexAI 200k token window, the request fails entirely. The issue persists if I remove the VertexAI model and have only the Anthropic API model wrapped in FallbackModel

If I unwrap the Anthropic API model definition (as shown below) from being inside FallbackModel it works going beyond the 200k token window.

Example Code

from pydantic_ai import Agent
from pydantic_ai.models.anthropic import AnthropicModel, AnthropicModelSettings
from pydantic_ai.providers.anthropic import AnthropicProvider

claude_sonnet_4_model = AnthropicModel(
    "claude-sonnet-4-20250514",
    provider=AnthropicProvider(
        anthropic_client=anthropic_client,
    ),
    settings=AnthropicModelSettings(
        extra_headers={
            "anthropic-beta": "context-1m-2025-08-07"
        }
    )
)

agent = Agent(
  model=claude_sonnet_4_model,
  # If the model is wrapped in a FallbackModel, the `anthropic-beta` header isn't used
  # model=FallbackModel(claude_sonnet_4_model),
  model_settings=AnthropicModelSettings(
    temperature=1.0, max_tokens=1024
  )
)

Python, Pydantic AI & LLM client version

pydantic-ai 0.6.2
Python 3.12

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions