AG UI: Validation errors when using reasoning models with Ollama (gpt-oss and qwen3)

[celeritatem]

Initial Checks

• I confirm that I'm using the latest version of Pydantic AI
• I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue

Description

I am encountering some validation errors when running ag-ui with Ollama running two different reasoning models: gpt-oss:20b and qwen3:4b.

I am trying to implement the simplest examples from pydantic-ai's ag-ui docs (https://ai.pydantic.dev/ag-ui/#installation) and the simplest chat with your agent example from the ConvexKit quickstart (https://docs.copilotkit.ai/pydantic-ai/quickstart/pydantic-ai?path=exiting-agent).

I am completely new to pydantic-ai and ag-ui so its possible I might just be making an obvious mistake. If that's the case, please point me in the correct direction. Thanks.

gpt-oss:20b

I get the following error:

File ".../.venv/lib/python3.12/site-packages/pydantic_ai/ag_ui.py", line 366, in run_ag_ui
    raise e
  File ".../.venv/lib/python3.12/site-packages/pydantic_ai/ag_ui.py", line 356, in run_ag_ui
    async for event in _agent_stream(run):
  File ".../.venv/lib/python3.12/site-packages/pydantic_ai/ag_ui.py", line 390, in _agent_stream
    async for msg in _handle_model_request_event(stream_ctx, agent_event):
  File ".../.venv/lib/python3.12/site-packages/pydantic_ai/ag_ui.py", line 471, in _handle_model_request_event
    yield TextMessageContentEvent(
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File .../.venv/lib/python3.12/site-packages/pydantic/main.py", line 253, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for TextMessageContentEvent
delta
  String should have at least 1 character [type=string_too_short, input_value='', input_type=str]

The example worked as expected after I modified the TextMessageContentEvent instantiation around line 470 in the _handle_model_request_event function in ag_ui.py as follows,

            # Orig:
            # yield TextMessageContentEvent(
            #     message_id=stream_ctx.message_id,
            #     delta=delta.content_delta,
            # )
            if len(delta.content_delta) > 0:
                yield TextMessageContentEvent(
                    message_id=stream_ctx.message_id,
                    delta=delta.content_delta,
                )

qwen3:4b

I tried the same example with a different reasoning model to confirm the bug and I got a different error:

 File ".../.venv/lib/python3.12/site-packages/pydantic_ai/ag_ui.py", line 366, in run_ag_ui
    |     raise e
    |   File ".../.venv/lib/python3.12/site-packages/pydantic_ai/ag_ui.py", line 356, in run_ag_ui
    |     async for event in _agent_stream(run):
    |   File ".../.venv/lib/python3.12/site-packages/pydantic_ai/ag_ui.py", line 390, in _agent_stream
    |     async for msg in _handle_model_request_event(stream_ctx, agent_event):
    |   File ".../.venv/lib/python3.12/site-packages/pydantic_ai/ag_ui.py", line 459, in _handle_model_request_event
    |     yield ThinkingTextMessageContentEvent(
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File ".../.venv/lib/python3.12/site-packages/pydantic/main.py", line 253, in __init__
    |     validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | pydantic_core._pydantic_core.ValidationError: 1 validation error for ThinkingTextMessageContentEvent
    |   Value error, Delta must not be an empty string [type=value_error, input_value={'type': <EventType.THINK..._CONTENT'>, 'delta': ''}, input_type=dict]

There is a comment above line 459 about always sending empty content which confuses me because it seems to contradict the validation check in the ThinkingTextMessageContentEvent:

           # Always send the content even if it's empty, as it may be
            # used to indicate the start of thinking.
            yield ThinkingTextMessageContentEvent(
                type=EventType.THINKING_TEXT_MESSAGE_CONTENT,
                delta=part.content,
            )

class ThinkingTextMessageContentEvent(BaseEvent):
  ...
    def model_post_init(self, __context):
        if len(self.delta) == 0:
            raise ValueError("Delta must not be an empty string")

If I ignore the comment and add a delta length check as follows, I then get the same validation error I got with gpt-oss:20b. But, if I add another length check before instantiating the event the agent simply doesn't respond causing an error in the frontend Cannot send 'THINKING_TEXT_MESSAGE_START' event: A thinking step is not in progress. Create one with 'THINKING_START' first..

           # modification to line 459 _handle_model_request_event function in ag_ui.py
           if len(part.content) > 0:
                yield ThinkingTextMessageContentEvent(
                    type=EventType.THINKING_TEXT_MESSAGE_CONTENT,
                    delta=part.content,
                )

Example Code

from fastapi import FastAPI
from pydantic_ai import Agent
from pydantic_ai.ag_ui import handle_ag_ui_request
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.ollama import OllamaProvider
from starlette.requests import Request
from starlette.responses import Response

# Model:
model_name = "qwen3:4b" # OR, "gpt-oss:20b"

model = OpenAIChatModel(
    model_name=model_name,
    provider=OllamaProvider(base_url="http://127.0.0.1:11434/v1"),
)

agent = Agent(model)

# app = agent.to_ag_ui()

app = FastAPI()

@app.post("/")
async def run_agent(req: Request) -> Response:
    return await handle_ag_ui_request(agent, req)

Python, Pydantic AI & LLM client version

Python 3.12.3
pydantic v2.11.7
pydantic-ai-slim[ag-ui, openai] v0.8.1
ag-ui-protocol v0.1.8

Ollama + LLMs:
---------------
ollama 0.11.6
qwen3:4b             e55aed6fe643
gpt-oss:20b          aa4295ac10c3

JS Client:
-----------
"@copilotkit/react-core": "^1.10.3",
"@copilotkit/react-ui": "^1.10.3",
"@copilotkit/runtime": "^1.10.3",

[n1okr]

i was facing the same issue with anthropic thinking models. had to do a monkeypatch to fix it.
not sure that the code is not a bs, as it is an ai slop from cursor, but it works.

"""Monkeypatch for AG-UI protocol to fix thinking event issues with pydantic-ai.

This patch addresses three issues:
1. ThinkingTextMessageContentEvent validation that rejects empty strings
2. Missing THINKING_START and THINKING_END wrapper events in pydantic-ai
3. ThinkingPartDelta events not being properly wrapped with START/END
"""

from typing import AsyncGenerator, Any
import asyncio
import sys


def patch_ag_ui_thinking_events():
    """Apply monkeypatch to fix thinking event issues."""
    try:
        # Import the AG-UI events module first
        import ag_ui.core.events
        from ag_ui.core import EventType
        from ag_ui.core.events import ThinkingStartEvent, ThinkingEndEvent
        
        # Patch the ThinkingTextMessageContentEvent class in ag_ui BEFORE pydantic_ai imports it
        # Replace the model_post_init method that does the validation
        def no_validation_post_init(self, __context):
            pass  # Skip validation - allow empty strings
        
        # Apply the patch to the AG-UI class
        ag_ui.core.events.ThinkingTextMessageContentEvent.model_post_init = no_validation_post_init
        
        # Also check if pydantic_ai has already imported it
        if 'pydantic_ai.ag_ui' in sys.modules:
            # It's already imported, patch it there too
            import pydantic_ai.ag_ui
            if hasattr(pydantic_ai.ag_ui, 'ThinkingTextMessageContentEvent'):
                pydantic_ai.ag_ui.ThinkingTextMessageContentEvent.model_post_init = no_validation_post_init
        
        # Import pydantic_ai if not already imported
        import pydantic_ai.ag_ui
        
        # Wrap the entire agent stream to fix thinking events properly
        original_agent_stream = pydantic_ai.ag_ui._agent_stream
        
        async def patched_agent_stream(run) -> AsyncGenerator[Any, None]:
            """Wrapped agent stream that fixes thinking events."""
            thinking_active = False
            thinking_text_active = False
            
            async for event in original_agent_stream(run):
                # Handle thinking events
                if hasattr(event, 'type'):
                    # Track thinking text message state
                    if event.type == EventType.THINKING_TEXT_MESSAGE_START:
                        # Emit THINKING_START if needed
                        if not thinking_active:
                            thinking_active = True
                            yield ThinkingStartEvent(type=EventType.THINKING_START)
                        thinking_text_active = True
                        yield event
                        
                    elif event.type == EventType.THINKING_TEXT_MESSAGE_CONTENT:
                        # Ensure we have started thinking text
                        if not thinking_text_active:
                            # Start both thinking and text if needed
                            if not thinking_active:
                                thinking_active = True
                                yield ThinkingStartEvent(type=EventType.THINKING_START)
                            thinking_text_active = True
                            yield pydantic_ai.ag_ui.ThinkingTextMessageStartEvent(
                                type=EventType.THINKING_TEXT_MESSAGE_START
                            )
                        
                        # Skip empty delta events to avoid validation errors
                        if hasattr(event, 'delta') and event.delta:
                            yield event
                        # Don't yield empty delta events - just skip them
                            
                    elif event.type == EventType.THINKING_TEXT_MESSAGE_END:
                        yield event
                        thinking_text_active = False
                        # Emit THINKING_END if this was the last thinking event
                        if thinking_active:
                            thinking_active = False
                            yield ThinkingEndEvent(type=EventType.THINKING_END)
                    else:
                        # Pass through all other events
                        yield event
                else:
                    # Pass through non-typed events
                    yield event
        
        # Replace the stream function
        pydantic_ai.ag_ui._agent_stream = patched_agent_stream
        
        print("✅ AG-UI monkeypatch applied:")
        print("   - Empty thinking event deltas are now allowed")
        print("   - Missing THINKING_START/END events are now added")
        
    except Exception as e:
        print(f"⚠️  Failed to apply AG-UI monkeypatch: {e}")
        print("   Thinking mode may not work properly")

[DouweM]

@celeritatem Thanks for the report. I already fixed the thinking issue in #2754. For the text part, adding if part.content_delta: (which is equivalent to if len(part.content_delta) > 0: should do it -- would you mind filing a PR for that?

[celeritatem]

@DouweM

You're welcome.

Here's the PR #2779

I want to play around with UIs for agents + local models so I wouldn't mind helping out more, although I don't understand the code all that well.