Allow empty response

[dconathan]

Initial Checks

• I confirm that I'm using the latest version of Pydantic AI
• I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue

Description

There are use cases where it is legitimate for the model to return an empty response. Currently this causes agents to crash in unintuitive ways (Pydantic AI retries the request and then the retry request fails with a Bad Request error). In prior versions this just raised a UnexpectedModelBehavior: Received empty model response

Relevant retry code is here:

pydantic-ai/pydantic_ai_slim/pydantic_ai/_agent_graph.py

Lines 546 to 549 in dc3c9e9

	if not self.model_response.parts:
	# we got an empty response.
	# this sometimes happens with anthropic (and perhaps other models)
	# when the model has already returned text along side tool calls

Current behavior causes the request to retry which then for me results in a BadRequestError:

AzureException BadRequestError - Invalid value for 'content': expected a string, got null.

An example "legitimate" use case is if you have a "relevant text extractor" agent and it is instructed to only extract relevant verbatim text from the input and to return no text if there is no relevant text (see example).

Currently an easy workaround is to prompt the model to return the text "no relevant text" instead of an empty string - but this feels like a hacky workaround.

Is Pydantic AI intentionally making the opinionated choice that "there is no valid reason a model could return an empty response" ? What if I have fine-tuned a model to respond with empty strings in certain scenarios? Should we have an opt in like allow_empty_response=True in model or agent settings?

Example Code

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider


agent = Agent(
    model=OpenAIChatModel(
        "azure/gpt-5-nano",
        provider=OpenAIProvider(
            base_url="...",
        ),
    ),
    instructions="You are a text extractor agent. You always respond with verbatim text from the provided context that is relevant to George Washington. If there is no relevant text, respond with an empty string. Do not provide any additional commentary or information.",
)

text = """Taylor Alison Swift (born December 13, 1989) is an American singer-songwriter. Known for her autobiographical songwriting, artistic reinventions, and cultural impact, Swift is the highest-grossing live music artist, the wealthiest female musician, and one of the best-selling music artists of all time.

Swift signed with Big Machine Records in 2005 and debuted as a country singer with the albums Taylor Swift (2006) and Fearless (2008). The singles "Teardrops on My Guitar", "Love Story", and "You Belong with Me" found crossover success on country and pop radio formats. Speak Now (2010) expanded her country pop sound with rock influences, and Red (2012) featured a pop-friendly production. She recalibrated her artistic identity from country to pop with the synth-pop album 1989 (2014); ensuing media scrutiny inspired the hip-hop-imbued Reputation (2017). Through the 2010s, she accumulated the US Billboard Hot 100 number-one singles "We Are Never Ever Getting Back Together", "Shake It Off", "Blank Space", "Bad Blood", and "Look What You Made Me Do"."""


agent.run_sync(text)

Python, Pydantic AI & LLM client version

Python 3.13.7
Pydantic 2.11.10
Pydantic AI 1.0.15
OpenAI 2.2.0

Backend is Azure OpenAI endpoint through a LiteLLM proxy server.

[DouweM]

@dconathan Thanks for the report. We do indeed consider empty responses invalid, based on issues with APIs like Gemini sometimes not returning anything until the request is retried. Generally, we're of the opinion that "Generative AI" models should generate, and that even the case of "nothing to extract" should be represented by an explicitly-negative generation instead of just the absence of an explicitly-positive one.

In other words, I recommend using structured data types to encapsulate the ExtractedText and NoTextToExtract cases, e.g.:

@dataclass
class ExtractedText:
    text: str

@dataclass
class NoTextToExtract:
    pass


agent = Agent(
    model=OpenAIChatModel(
        "azure/gpt-5-nano",
        provider=OpenAIProvider(
            base_url="...",
        ),
    ),
    output_type=ExtractedText | NoTextToExtract
)

You could also remove NoTextToExtract and make ExtractedText.text optional (str | None), and check the presence of that field.

What if I have fine-tuned a model to respond with empty strings in certain scenarios? Should we have an opt in like allow_empty_response=True in model or agent settings?

I believe it's generally harder to fine-tune generative models to not generate anything, than to generate some kind of special negative response token, but I agree that if you did have such a model, Pydantic AI currently wouldn't handle it correctly.

Is the workaround above acceptable or do you really need a mode where fully empty responses and/or responses with a text part with part.text == '' are treated as valid text output?

[dconathan]

That's fine I just wanted to confirm that this is intended behavior - I know about the possible workarounds. I just wanted to report this in case this wasn't intended behavior, and/or if you want to change the error-raising behavior. I.e., in my case the retry failed with a bad request error which does not seem like desirable behavior.

[DouweM]

in my case the retry failed with a bad request error which does not seem like desirable behavior.

@dconathan Yeah I agree. Could you share the stack trace for that error, and if possible the HTTP request body that caused it? (See https://ai.pydantic.dev/logfire)

[clhodapp]

A bad pattern that we seem to be seeing with Gemini (at least as well as I can tell from traces):

1. We send a request to a PydanticAI agent with some tools and a string output type (in order to get response streaming back to the client via ag-ui)
2. PydanticAI calls Gemini
3. Gemini returns both a message to show to the user and a tool call request
4. PydanticAI invokes the indicated tool function and calls Gemini again with the tool result
5. Gemini returns an empty-string response, as if to say "We're done here. Nothing else is needed."
6. Pydantic doesn't like the empty response and calls Gemini again with a validation error message, hoping to get a non-empty result
7. Gemini seemingly concludes that the tool call didn't work for some reason and returns a new client-destined message saying sorry and a fresh tool call to the same tool
8. PydanticAI invokes the tool function again and calls Gemini again with the tool result
9. Gemini again returns an empty reply (from its perspective, we're done here once again)
10. PydanticAI again dislikes this result and the cycle begins anew

This happens until our tool call retries have been exhausted.

[DouweM]

@clhodapp Can you see if the problem is solved with #3360 (which is already in main and I'll get out in a release later today)?

That should get us into this branch where we'll use the text from the most recent response (that had the tool calls) as output:

pydantic-ai/pydantic_ai_slim/pydantic_ai/_agent_graph.py

Lines 570 to 593 in 86b645f

	if not self.model_response.parts:
	# we got an empty response.
	# this sometimes happens with anthropic (and perhaps other models)
	# when the model has already returned text along side tool calls
	if text_processor := output_schema.text_processor: # pragma: no branch
	# in this scenario, if text responses are allowed, we return text from the most recent model
	# response, if any
	for message in reversed(ctx.state.message_history):
	if isinstance(message, _messages.ModelResponse):
	text = ''
	for part in message.parts:
	if isinstance(part, _messages.TextPart):
	text += part.content
	elif isinstance(part, _messages.BuiltinToolCallPart):
	# Text parts before a built-in tool call are essentially thoughts,
	# not part of the final result output, so we reset the accumulated text
	text = '' # pragma: no cover
	if text:
	try:
	self._next_node = await self._handle_text_response(ctx, text, text_processor)
	return
	except ToolRetryError:
	# If the text from the preview response was invalid, ignore it.
	pass

[clhodapp]

@DouweM The bad pattern I described seems to be resolved in Pydantic AI v1.14.0