Rare Assertion error when using Gemini Models in streaming mode

[amiyapatanaik]

Initial Checks

• I confirm that I'm using the latest version of Pydantic AI
• I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue

Description

For a long time, random assertion errors continued to happen for complex agents using Gemini models in streaming mode. It is rare but happens enough times to make the system unreliable. The error as logged in Logfire before the 0.7.2 update was consistently:

Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/opentelemetry/trace/__init__.py", line 589, in use_span
    yield span
  File "/opt/venv/lib/python3.12/site-packages/pydantic_graph/graph.py", line 260, in iter
    yield GraphRun[StateT, DepsT, RunEndT](
  File "/opt/venv/lib/python3.12/site-packages/pydantic_ai/agent.py", line 754, in iter
    yield agent_run
  File "/opt/venv/lib/python3.12/site-packages/nxd/chat.py", line 198, in handle_agent_streaming
    async for event in request_stream:
  File "/opt/venv/lib/python3.12/site-packages/pydantic_ai/result.py", line 162, in aiter
    async for event in usage_checking_stream:
  File "/opt/venv/lib/python3.12/site-packages/pydantic_ai/models/google.py", line 455, in _get_event_iterator
    assert candidate.content.parts is not None
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

The 0.7.2 patch did not solve the issue:

Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/opentelemetry/trace/__init__.py", line 589, in use_span
    yield span
  File "/opt/venv/lib/python3.12/site-packages/pydantic_graph/graph.py", line 259, in iter
    yield GraphRun[StateT, DepsT, RunEndT](
  File "/opt/venv/lib/python3.12/site-packages/pydantic_ai/agent/__init__.py", line 661, in iter
    yield agent_run
  File "/opt/venv/lib/python3.12/site-packages/nxd/chat.py", line 198, in handle_agent_streaming
    async for event in request_stream:
  File "/opt/venv/lib/python3.12/site-packages/pydantic_ai/result.py", line 441, in _iterator
    async for item in stream_response:
  File "/opt/venv/lib/python3.12/site-packages/pydantic_ai/models/__init__.py", line 575, in iterator_with_final_event
    async for event in iterator:
  File "/opt/venv/lib/python3.12/site-packages/pydantic_ai/models/google.py", line 532, in _get_event_iterator
    assert candidate.content.parts is not None
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

The Root Cause

• Streaming API Behavior: When you stream a response from a model like Gemini, you get a series of "chunks." Most of these chunks contain content deltas (e.g., the next few words of text). The very last chunk, however, often contains no new content but instead provides final metadata, such as the finish_reason (e.g., "STOP", "MAX_TOKENS") and the final usage_metadata.
• The Problematic Chunk: It appears that this final chunk from the Gemini API can be structured with a candidate.content object that exists, but its parts attribute is None. This signifies "there is a content block, but it has no new parts in this specific chunk."

This is separate from the issue when sometime Gemini models stream thoughts and then forget to actually stream the answer #2469 .

My suggestion is to change:

# ... inside _get_event_iterator
            assert candidate.content.parts is not None
            for part in candidate.content.parts:
                if part.text is not None:
# ...

to

# ... inside _get_event_iterator
            for part in candidate.content.parts or []:
                if part.text is not None:
# ...

Interestingly, the non-streaming response handler already does it this way:

# in GoogleModel._process_response (for non-streaming)

parts = response.candidates[0].content.parts or []
# This is the correct, safe way to handle it.

Example Code

Python, Pydantic AI & LLM client version

All Gemini Models

[lucasrodes]

i just encountered this issue myself. Issue closed and resolved "34 minutes ago". This feels bleeding edge.
thanks everyone for your hard work at pydantic.

[ArneZsng]

Came here to see if this issue (been encountering it every now and then since last week) was experienced by someone else. Found that it was solved not even an hour ago. Amazing.

[osczen]

Have been seeing this issue with Gemini models for a long time as well, thanks for the fix!

[amiyapatanaik]

I was wondering why no one has reported this issue! Glad it is finally resolved.

[ArneZsng]

After investigating the issue further, the fix in #2618 seems incomplete, given that the condition of candidate.content.parts is None and the candidate.finish_reason == 'STOP' is actually expected behavior. In such a case, the raise UnexpectedModelBehavior('Content field missing from streaming Gemini response', str(chunk)) overshadows the assignment of parts = candidate.content.parts or [].

I added my attempt to fix this in #2631.

A similar issue is mentioned in #2630.

[amiyapatanaik]

@ArneZsng If I'm not wrong, this affects the tests not the actual gemini model code within pydantic? I'm asking because we are not updating to the latest build as it will break certain compatibilities.

[ArneZsng]

@amiyapatanaik My bad, the wrong PR was linked. It should have been #2631 (I corrected the link above).

Please let me know if your question still persists after looking at the correctly linked PR.