ChatCompletionService giving out error when using hugging face router with providers

[JayGhiya]

Initial Checks

• I confirm that I'm using the latest version of Pydantic AI
• I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue

Description

Invalid response from OpenAI chat completions endpoint: 1 validation error for ChatCompletion\nservice_tier\n Input should be 'auto', 'default', 'flex', 'scale' or 'priority' [type=literal_error, input_value='on_demand', input_type=str]\n For further information visit https://errors.pydantic.dev/2.11/v/literal_error

Example Code

from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.providers.openai import OpenAIProvider


model = OpenAIModel('moonshotai/Kimi-K2-Instruct:groq', provider=OpenAIProvider(base_url="https://router.huggingface.co/v1",api_key=os.environ["HF_TOKEN"]))

Python, Pydantic AI & LLM client version

Python: python 3.13
Pydantic AI: 0.6.2

[JayGhiya]

Currently i monkey patch to make it work:

from typing_extensions import Literal
from typing import Optional
from openai.types.chat.chat_completion import ChatCompletion
import pydantic


def patch_openai_service_tier() -> None:
    """Monkey patch OpenAI ChatCompletion to accept 'on_demand' service_tier.
    
    The OpenAI API specification allows additional service_tier values beyond
    the ones defined in the Python client. This patch extends the validation
    to include 'on_demand' which is used by some compatible providers.
    """
    # Get the current field info for service_tier
    current_field = ChatCompletion.model_fields.get('service_tier')
    if current_field is None:
        return
    
    # Create new field info with extended Literal types
    extended_annotation = Optional[Literal["auto", "default", "flex", "scale", "priority", "on_demand"]]
    
    # Update the field annotation
    new_field = pydantic.fields.FieldInfo(
        annotation=extended_annotation,
        default=current_field.default,
        description=current_field.description,
    )
    
    # Update the model fields and rebuild
    ChatCompletion.model_fields['service_tier'] = new_field
    ChatCompletion.model_rebuild()

[Kludex]

We now have a HuggingFaceModel: https://ai.pydantic.dev/models/huggingface/.

Please try that.

[JayGhiya]

will report back post trying hugging face model.

[Solarden]

I'm seeing the same validation failure, but with AWS Bedrock’s OpenAI-compatible shim instead of Hugging Face router.

Their responses return:

"service_tier": "standard"

which isn’t in the Literal["auto","default","flex","scale","priority"] set. As a result, OpenAIChatModel fails with the exact same error:

pydantic_ai.exceptions.UnexpectedModelBehavior: Invalid response from OpenAI chat completions endpoint:
1 validation error for ChatCompletion
service_tier
  Input should be 'auto', 'default', 'flex', 'scale' or 'priority'
  [type=literal_error, input_value='standard', input_type=str]

Minimal repro (no external API call)

from pydantic_ai.models import openai as pyd_openai

fake_response = {
    "id": "chatcmpl-123",
    "object": "chat.completion",
    "created": 123456789,
    "model": "gpt-4o-mini",
    "choices": [
        {"index": 0, "message": {"role": "assistant", "content": "Hello!"}, "finish_reason": "stop"}
    ],
    "usage": {"prompt_tokens": 5, "completion_tokens": 7, "total_tokens": 12},
    "service_tier": "standard",  # Bedrock-specific value
}

pyd_openai.chat.ChatCompletion.model_validate(fake_response)

Why this matters

• Hugging Face returns on_demand
• Bedrock returns standard
• Both are OpenAI-compatible providers, and both currently break validation.

Workaround

I applied the same monkey patch suggested above, but extended it to allow "standard" as well (or changed the field to Optional[str]).

Suggestion

Instead of hard-failing on unknown service_tier values, it might be better to:

• Accept any string (Optional[str]), or
• Provide a hook to normalize/transform provider-specific values before validation.

---

Happy to open a separate issue if maintainers prefer, but since it’s the exact same root cause (strict validation of service_tier), I thought it best to consolidate here.