Gemma models used through OpenRouter do not support Structured Output

[JuanCruzC97]

Initial Checks

• I confirm that I'm using the latest version of Pydantic AI
• I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue

Description

When using gemma models through OpenRouter I get 3 different errors when trying to use structured output with the different approaches pydantic ai offers.

1. When calling the first option in the code example I get: ModelHTTPError: status_code: 404, model_name: google/gemma-3-27b-it:free, body: {'message': 'No endpoints found that support tool use. To learn more about provider routing, visit: https://openrouter.ai/docs/provider-routing', 'code': 404}

2. When calling the second option I get: ModelHTTPError: status_code: 400, model_name: google/gemma-3-27b-it:free, body: {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{\n "error": {\n "code": 400,\n "message": "JSON mode is not enabled for models/gemma-3-27b-it",\n "status": "INVALID_ARGUMENT"\n }\n}\n', 'provider_name': 'Google AI Studio'}}

3. When calling the third option I get: ModelHTTPError: status_code: 400, model_name: google/gemma-3-27b-it:free, body: {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{\n "error": {\n "code": 400,\n "message": "Developer instruction is not enabled for models/gemma-3-27b-it",\n "status": "INVALID_ARGUMENT"\n }\n}\n', 'provider_name': 'Google AI Studio'}}

I wanted to confirm if I'm doing something wrong or there's no way to use the free gemma3 models from openrouter with pydantic ai structured outputs.

Example Code

from pydantic import BaseModel

from pydantic_ai import Agent, NativeOutput, PromptedOutput
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openrouter import OpenRouterProvider

import nest_asyncio
nest_asyncio.apply()

MODEL = "google/gemma-3-27b-it:free"

model = OpenAIChatModel(
    MODEL,
    provider=OpenRouterProvider(api_key=API_KEY),
)

class Ent(BaseModel):
    text: str
    ner: Literal['ORG', 'PER', 'LOC', 'MISC']

class EntList(BaseModel):
    entities: list[Ent]

# Option 1:
agent = Agent(model, output_type=EntList)

# Option 2:
agent = Agent(
    model, 
    output_type=NativeOutput(
        EntList, 
        name='List of Named Entities Detected',
        description='Return a list of detected entities in the text.'
    ),

)

# Option 3:
agent = Agent(
    model, 
    output_type=PromptedOutput(
        EntList, 
        name='List of Named Entities Detected',
        description='Return a list of detected entities in the text.'
    ),

)

result = agent.run_sync(BASE_INSTRUCTION + f"El texto a procesar es: {processed_data[0].paragraph}")

Python, Pydantic AI & LLM client version

Python 3.12.11
Pydantic AI 1.0.10
Open Router using "google/gemma-3-27b-it:free" model

[moritzwilksch]

None of these are pydantic-ai issues.

1. openrouter cannot find a provider that support tool calling because the models were trained without a standardized tool calling scaffolding. See official guide from google: The way they to tool-calling is entirely free-form prompting.
2. The error very clearly states that the provider doesn't support structured outputs for this model.
3. you should note that the Gemma models were trained without explicit system instructions, so passing "role": "system" is not supported.

Conclusion: Use a different model if you want any of 1) standardized tool calling, 2) structured output or 3) actual system instructions.

[JuanCruzC97]

Great! Will try with other models, thanks!