Content Filtering Exception Handling

[mscherrmann]

I noticed two issues with the Content Filtering behavior of Pydantic AI when used with Azure OpenAI:

1. Inconsistent error handling for content filtering: Azure OpenAI uses two different types of content filters:

   • Prompt filter: Azure raises a BadRequestError PromptFilterBadRequestBody.txt
   • Completion filter: The finish_reason on the response will be content_filter (CompletionFilter.txt)

   (Reference: Azure OpenAI Content Filter documentation)

   This translates to two different errors in Pydantic AI:

   • For prompt filtering: ModelHTTPError
   • For completion filtering: UnexpectedModelBehavior error. Here we don't even see the filter type.

2. Lack of model-agnostic content filter handling: Different providers handle content filtering differently. For example, Vertex AI handles content filters differently, which makes the current content filter handling in Pydantic AI not model-agnostic.

Proposed Solution

1. Implement a unique content filter exception that can be used to handle only these specific cases. This exception should:

   • Contain filter type information (hate, sexual, violence, and self-harm) to help users identify problematic and sensitive information in their prompts
   • Include child exceptions (prompt filter and completion filter) to help users handle these cases separately if needed
   • Provide a consistent interface for handling content filtering errors
   • If relevant, make sure to not leake any private user information contained in prompt or generation

2. Ensure that Pydantic AI's behavior on content filters is consistent across different model providers, making the library truly model-agnostic.

These improvements would allow users to better handle content filtering scenarios, potentially by altering prompts or model inputs when content filtering is triggered.

References

No response

[mscherrmann]

Here is a MRE for both cases. The prompts are examples taken from the Azure page:

from pydantic_ai import Agent

SYSTEM_PROMPT = "You are a helpful assistant."
PROMPT_FILTER = "[Identity Group] threaten our existence, so we should wipe them out."
GENERATION_FILTER = "He was lying in his bed drenched in blood with an absolute look of horror on his face."

if __name__ == "__main__":
    """Run the script."""
    agent = Agent(
        "azure:gpt-4o-mini",
        system_prompt=SYSTEM_PROMPT,
        model_settings={"top_p": 0.0},
    )
    await agent.run(
        user_prompt=GENERATION_FILTER
    )  # Change the user prompt to test different scenarios

[tostenzel]

@samuelcolvin This is essential to many applications (just google content filter). E.g. the OpenAI content filter leans heavily towards false positives. In our chat bot applications these problems occur quite often.

In my opinion, the proposed solution should have a high priority, since this affects the core of pydantic-ai's requirements, being model agnostic and reliable.

At the moment, we have to wrap pydantic-ai (related to this issue) and handle the content filter related errors on our own for each model provider in order to not get surprised by the behavior triggerd by the content filter with misleading errors!

We would be very happy If you could help us here :-)

[wpnbos]

Bump

[DouweM]

@wpnbos Appreciate the bump!

I agree we should handle this consistently across models.

A number of models return a regular response with a finish_reason field that we then map and store on ModelResponse.finish_reason as 'content_filter', but we don't currently do anything with that information, we just continue processing as normal, possibly retrying the request if the response was completely empty. GoogleModel is the exception, which does raise an UnexpectedModelBehavior error in this case:

pydantic-ai/pydantic_ai_slim/pydantic_ai/models/google.py

Lines 470 to 473 in 59faf42

	if finish_reason == 'content_filter' and raw_finish_reason:
	raise UnexpectedModelBehavior(
	f'Content filter {raw_finish_reason.value!r} triggered', response.model_dump_json()
	)

Models that return an HTTP error (or model SDKs that raise an error) are handled by raising our ModelHTTPError, but again without a special type to indicate that it was the result of a content filter.

Since retrying is unlikely to work, a dedicated exception when ModelResponse.finish_reason == 'content_filter' makes sense to me, in addition to handling content filter API/SDK errors.

@wpnbos Just so we know where to start, which model providers/APIs are you primarily interested in seeing this fixed for?

[wpnbos]

@DouweM a common exception for hitting a content filter, as you describe, would indeed be a great thing to have.

Seeing this fixed for any model provider that serves Anthropic models (Vertex/GCP, Bedrock, Anthropic) would be most valuable.

Thank for being on top of this!