coverage

Author: DouweM

docs/builtin-tools.md

@@ -31,7 +31,7 @@ making it ideal for queries that require up-to-date data.
 |----------|-----------|-------|
 | OpenAI Responses | ✅ | Full feature support. To include search results on the [`BuiltinToolReturnPart`][pydantic_ai.messages.BuiltinToolReturnPart] that's available via [`ModelResponse.builtin_tool_calls`][pydantic_ai.messages.ModelResponse.builtin_tool_calls], enable the [`OpenAIResponsesModelSettings.openai_include_web_search_sources`][pydantic_ai.models.openai.OpenAIResponsesModelSettings.openai_include_web_search_sources] [model setting](agents.md#model-run-settings). |
 | Anthropic | ✅ | Full feature support |
-| Google | ✅ | No parameter support. No [`BuiltinToolCallPart`][pydantic_ai.messages.BuiltinToolCallPart] or [`BuiltinToolReturnPart`][pydantic_ai.messages.BuiltinToolReturnPart] is generated when streaming. Using built-in tools and user tools (including [output tools](output.md#tool-output)) at the same time is not supported; to use structured output, use [`PromptedOutput`](output.md#prompted-output) instead. |
+| Google | ✅ | No parameter support. No [`BuiltinToolCallPart`][pydantic_ai.messages.BuiltinToolCallPart] or [`BuiltinToolReturnPart`][pydantic_ai.messages.BuiltinToolReturnPart] is generated when streaming. Using built-in tools and function tools (including [output tools](output.md#tool-output)) at the same time is not supported; to use structured output, use [`PromptedOutput`](output.md#prompted-output) instead. |
 | Groq | ✅ | Limited parameter support. To use web search capabilities with Groq, you need to use the [compound models](https://console.groq.com/docs/compound). |
 | OpenAI Chat Completions | ❌ | Not supported |
 | Bedrock | ❌ | Not supported |
@@ -123,7 +123,7 @@ in a secure environment, making it perfect for computational tasks, data analysi
 | Provider | Supported | Notes |
 |----------|-----------|-------|
 | OpenAI | ✅ | To include code execution output on the [`BuiltinToolReturnPart`][pydantic_ai.messages.BuiltinToolReturnPart] that's available via [`ModelResponse.builtin_tool_calls`][pydantic_ai.messages.ModelResponse.builtin_tool_calls], enable the [`OpenAIResponsesModelSettings.openai_include_code_execution_outputs`][pydantic_ai.models.openai.OpenAIResponsesModelSettings.openai_include_code_execution_outputs] [model setting](agents.md#model-run-settings). If the code execution generated images, like charts, they will be available on [`ModelResponse.images`][pydantic_ai.messages.ModelResponse.images] as [`BinaryImage`][pydantic_ai.messages.BinaryImage] objects. The generated image can also be used as [image output](output.md#image-output) for the agent run. |
-| Google | ✅ | Using built-in tools and user tools (including [output tools](output.md#tool-output)) at the same time is not supported; to use structured output, use [`PromptedOutput`](output.md#prompted-output) instead. |
+| Google | ✅ | Using built-in tools and function tools (including [output tools](output.md#tool-output)) at the same time is not supported; to use structured output, use [`PromptedOutput`](output.md#prompted-output) instead. |
 | Anthropic | ✅ | |
 | Groq | ❌ | |
 | Bedrock | ❌ | |
@@ -315,7 +315,7 @@ allowing it to pull up-to-date information from the web.
 
 | Provider | Supported | Notes |
 |----------|-----------|-------|
-| Google | ✅ | No [`BuiltinToolCallPart`][pydantic_ai.messages.BuiltinToolCallPart] or [`BuiltinToolReturnPart`][pydantic_ai.messages.BuiltinToolReturnPart] is currently generated; please submit an issue if you need this. Using built-in tools and user tools (including [output tools](output.md#tool-output)) at the same time is not supported; to use structured output, use [`PromptedOutput`](output.md#prompted-output) instead. |
+| Google | ✅ | No [`BuiltinToolCallPart`][pydantic_ai.messages.BuiltinToolCallPart] or [`BuiltinToolReturnPart`][pydantic_ai.messages.BuiltinToolReturnPart] is currently generated; please submit an issue if you need this. Using built-in tools and function tools (including [output tools](output.md#tool-output)) at the same time is not supported; to use structured output, use [`PromptedOutput`](output.md#prompted-output) instead. |
 | OpenAI | ❌ | |
 | Anthropic | ❌ | |
 | Groq | ❌ | |

pydantic_ai_slim/pydantic_ai/_agent_graph.py

@@ -567,63 +567,67 @@ async def _run_stream(  # noqa: C901
             output_schema = ctx.deps.output_schema
 
             async def _run_stream() -> AsyncIterator[_messages.HandleResponseEvent]:  # noqa: C901
-                try:
-                    if not self.model_response.parts:
-                        # we got an empty response.
-                        # this sometimes happens with anthropic (and perhaps other models)
-                        # when the model has already returned text along side tool calls
-                        if text_processor := output_schema.text_processor:
-                            # in this scenario, if text responses are allowed, we return text from the most recent model
-                            # response, if any
-                            for message in reversed(ctx.state.message_history):
-                                if isinstance(message, _messages.ModelResponse):
-                                    text = ''
-                                    for part in message.parts:
-                                        if isinstance(part, _messages.TextPart):
-                                            text += part.content
-                                        elif isinstance(part, _messages.BuiltinToolCallPart):
-                                            # Text parts before a built-in tool call are essentially thoughts,
-                                            # not part of the final result output, so we reset the accumulated text
-                                            text = ''  # pragma: no cover
-                                    if text:
+                if not self.model_response.parts:
+                    # we got an empty response.
+                    # this sometimes happens with anthropic (and perhaps other models)
+                    # when the model has already returned text along side tool calls
+                    if text_processor := output_schema.text_processor:  # pragma: no branch
+                        # in this scenario, if text responses are allowed, we return text from the most recent model
+                        # response, if any
+                        for message in reversed(ctx.state.message_history):
+                            if isinstance(message, _messages.ModelResponse):
+                                text = ''
+                                for part in message.parts:
+                                    if isinstance(part, _messages.TextPart):
+                                        text += part.content
+                                    elif isinstance(part, _messages.BuiltinToolCallPart):
+                                        # Text parts before a built-in tool call are essentially thoughts,
+                                        # not part of the final result output, so we reset the accumulated text
+                                        text = ''  # pragma: no cover
+                                if text:
+                                    try:
                                         self._next_node = await self._handle_text_response(ctx, text, text_processor)
                                         return
+                                    except ToolRetryError:
+                                        # If the text from the preview response was invalid, ignore it.
+                                        pass
+
+                    # Go back to the model request node with an empty request, which means we'll essentially
+                    # resubmit the most recent request that resulted in an empty response,
+                    # as the empty response and request will not create any items in the API payload,
+                    # in the hope the model will return a non-empty response this time.
+                    ctx.state.increment_retries(ctx.deps.max_result_retries, model_settings=ctx.deps.model_settings)
+                    run_context = build_run_context(ctx)
+                    instructions = await ctx.deps.get_instructions(run_context)
+                    self._next_node = ModelRequestNode[DepsT, NodeRunEndT](
+                        _messages.ModelRequest(parts=[], instructions=instructions)
+                    )
+                    return
+
+                text = ''
+                tool_calls: list[_messages.ToolCallPart] = []
+                files: list[_messages.BinaryContent] = []
+
+                for part in self.model_response.parts:
+                    if isinstance(part, _messages.TextPart):
+                        text += part.content
+                    elif isinstance(part, _messages.ToolCallPart):
+                        tool_calls.append(part)
+                    elif isinstance(part, _messages.FilePart):
+                        files.append(part.content)
+                    elif isinstance(part, _messages.BuiltinToolCallPart):
+                        # Text parts before a built-in tool call are essentially thoughts,
+                        # not part of the final result output, so we reset the accumulated text
+                        text = ''
+                        yield _messages.BuiltinToolCallEvent(part)  # pyright: ignore[reportDeprecated]
+                    elif isinstance(part, _messages.BuiltinToolReturnPart):
+                        yield _messages.BuiltinToolResultEvent(part)  # pyright: ignore[reportDeprecated]
+                    elif isinstance(part, _messages.ThinkingPart):
+                        pass
+                    else:
+                        assert_never(part)
 
-                        # Go back to the model request node with an empty request, which means we'll essentially
-                        # resubmit the most recent request that resulted in an empty response,
-                        # as the empty response and request will not create any items in the API payload,
-                        # in the hope the model will return a non-empty response this time.
-                        ctx.state.increment_retries(ctx.deps.max_result_retries, model_settings=ctx.deps.model_settings)
-                        run_context = build_run_context(ctx)
-                        instructions = await ctx.deps.get_instructions(run_context)
-                        self._next_node = ModelRequestNode[DepsT, NodeRunEndT](
-                            _messages.ModelRequest(parts=[], instructions=instructions)
-                        )
-                        return
-
-                    text = ''
-                    tool_calls: list[_messages.ToolCallPart] = []
-                    files: list[_messages.BinaryContent] = []
-
-                    for part in self.model_response.parts:
-                        if isinstance(part, _messages.TextPart):
-                            text += part.content
-                        elif isinstance(part, _messages.ToolCallPart):
-                            tool_calls.append(part)
-                        elif isinstance(part, _messages.FilePart):
-                            files.append(part.content)
-                        elif isinstance(part, _messages.BuiltinToolCallPart):
-                            # Text parts before a built-in tool call are essentially thoughts,
-                            # not part of the final result output, so we reset the accumulated text
-                            text = ''
-                            yield _messages.BuiltinToolCallEvent(part)  # pyright: ignore[reportDeprecated]
-                        elif isinstance(part, _messages.BuiltinToolReturnPart):
-                            yield _messages.BuiltinToolResultEvent(part)  # pyright: ignore[reportDeprecated]
-                        elif isinstance(part, _messages.ThinkingPart):
-                            pass
-                        else:
-                            assert_never(part)
-
+                try:
                     # At the moment, we prioritize at least executing tool calls if they are present.
                     # In the future, we'd consider making this configurable at the agent or run level.
                     # This accounts for cases like anthropic returns that might contain a text response

pydantic_ai_slim/pydantic_ai/_output.py

@@ -35,7 +35,7 @@
 from .toolsets.abstract import AbstractToolset, ToolsetTool
 
 if TYPE_CHECKING:
-    from .profiles import ModelProfile
+    pass
 
 T = TypeVar('T')
 """An invariant TypeVar."""
@@ -384,12 +384,6 @@ def _build_processor(
 
         return UnionOutputProcessor(outputs=outputs, strict=strict, name=name, description=description)
 
-    def raise_if_unsupported(self, profile: ModelProfile) -> None:
-        """Raise an error if the mode is not supported by this model."""
-        # TODO (DouweM): Remove method?
-        if self.allows_image and not profile.supports_image_output:
-            raise UserError('Image output is not supported by this model.')
-
 
 @dataclass(init=False)
 class OutputSchemaWithoutMode(BaseOutputSchema[OutputDataT]):
@@ -439,10 +433,6 @@ def __init__(
     def mode(self) -> OutputMode | None:
         return 'text'
 
-    def raise_if_unsupported(self, profile: ModelProfile) -> None:
-        """Raise an error if the mode is not supported by this model."""
-        super().raise_if_unsupported(profile)
-
 
 class ImageOutputSchema(OutputSchema[OutputDataT]):
     def __init__(self, *, allows_deferred_tools: bool):
@@ -452,11 +442,6 @@ def __init__(self, *, allows_deferred_tools: bool):
     def mode(self) -> OutputMode | None:
         return 'image'
 
-    def raise_if_unsupported(self, profile: ModelProfile) -> None:
-        """Raise an error if the mode is not supported by this model."""
-        # This already raises if image output is not supported by this model.
-        super().raise_if_unsupported(profile)
-
 
 @dataclass(init=False)
 class StructuredTextOutputSchema(OutputSchema[OutputDataT], ABC):
@@ -479,11 +464,6 @@ class NativeOutputSchema(StructuredTextOutputSchema[OutputDataT]):
     def mode(self) -> OutputMode | None:
         return 'native'
 
-    def raise_if_unsupported(self, profile: ModelProfile) -> None:
-        """Raise an error if the mode is not supported by this model."""
-        if not profile.supports_json_schema_output:
-            raise UserError('Native structured output is not supported by this model.')
-
 
 @dataclass(init=False)
 class PromptedOutputSchema(StructuredTextOutputSchema[OutputDataT]):
@@ -522,11 +502,7 @@ def build_instructions(cls, template: str, object_def: OutputObjectDefinition) -
 
         return template.format(schema=json.dumps(schema))
 
-    def raise_if_unsupported(self, profile: ModelProfile) -> None:
-        """Raise an error if the mode is not supported by this model."""
-        super().raise_if_unsupported(profile)
-
-    def instructions(self, default_template: str) -> str:
+    def instructions(self, default_template: str) -> str:  # pragma: no cover
         """Get instructions to tell model to output JSON matching the schema."""
         template = self.template or default_template
         object_def = self.object_def
@@ -555,12 +531,6 @@ def __init__(
     def mode(self) -> OutputMode | None:
         return 'tool'
 
-    def raise_if_unsupported(self, profile: ModelProfile) -> None:
-        """Raise an error if the mode is not supported by this model."""
-        super().raise_if_unsupported(profile)
-        if not profile.supports_tools:
-            raise UserError('Tool output is not supported by this model.')
-
 
 class BaseOutputProcessor(ABC, Generic[OutputDataT]):
     @abstractmethod

pydantic_ai_slim/pydantic_ai/models/__init__.py

@@ -437,10 +437,11 @@ def prepare_request(
 
         if model_request_parameters.output_mode in ('native', 'prompted'):
             if not model_request_parameters.output_object:
-                raise UserError('An `output_object` is required when using `NativeOutput` or `PromptedOutput`.')
+                raise UserError(  # pragma: no cover
+                    'An `output_object` is required when using `NativeOutput` or `PromptedOutput`.'
+                )
 
             if model_request_parameters.output_mode == 'native' and not self.profile.supports_json_schema_output:
-                # TODO (DouweM): Call `NativeOutputSchema.raise_if_unsupported(self.profile)`?
                 raise UserError('Native structured output is not supported by this model.')
 
             if model_request_parameters.output_tools:
@@ -453,7 +454,7 @@ def prepare_request(
         else:
             if model_request_parameters.output_mode == 'tool':
                 if not model_request_parameters.output_tools and not model_request_parameters.function_tools:
-                    raise UserError('An `output_tools` list is required when using `ToolOutput`.')
+                    raise UserError('An `output_tools` list is required when using `ToolOutput`.')  # pragma: no cover
 
                 if not self.profile.supports_tools:
                     raise UserError('Tool output is not supported by this model.')
@@ -556,7 +557,7 @@ def _get_instructions(
             output_instructions = PromptedOutputSchema.build_instructions(
                 model_request_parameters.prompted_output_template, model_request_parameters.output_object
             )
-            if instructions is not None:
+            if instructions:
                 instructions = '\n\n'.join([instructions, output_instructions])
             else:
                 instructions = output_instructions

pydantic_ai_slim/pydantic_ai/models/anthropic.py

@@ -247,17 +247,14 @@ def prepare_request(
         settings = merge_model_settings(self.settings, model_settings)
         if (
             model_request_parameters.output_tools
-            and (
-                model_request_parameters.output_mode is None
-                or (model_request_parameters.output_mode == 'tool' and not model_request_parameters.allow_text_output)
-            )
             and settings
             and (thinking := settings.get('anthropic_thinking'))
             and thinking.get('type') == 'enabled'
         ):
             if model_request_parameters.output_mode is None:
                 model_request_parameters = replace(model_request_parameters, output_mode='prompted')
-            else:
+            elif model_request_parameters.output_mode == 'tool' and not model_request_parameters.allow_text_output:
+                # This would result in `tool_choice=required`, which Anthropic does not support with thinking.
                 raise UserError(
                     'Anthropic does not support thinking and output tools at the same time. Use `output_type=PromptedOutput(...)` instead.'
                 )

pydantic_ai_slim/pydantic_ai/models/function.py

@@ -135,6 +135,7 @@ async def request(
             allow_text_output=model_request_parameters.allow_text_output,
             output_tools=model_request_parameters.output_tools,
             model_settings=model_settings,
+            instructions=self._get_instructions(messages, model_request_parameters),
         )
 
         assert self.function is not None, 'FunctionModel must receive a `function` to support non-streamed requests'
@@ -168,6 +169,7 @@ async def request_stream(
             allow_text_output=model_request_parameters.allow_text_output,
             output_tools=model_request_parameters.output_tools,
             model_settings=model_settings,
+            instructions=self._get_instructions(messages, model_request_parameters),
         )
 
         assert self.stream_function is not None, (
@@ -216,6 +218,8 @@ class AgentInfo:
     """The tools that can called to produce the final output of the run."""
     model_settings: ModelSettings | None
     """The model settings passed to the run call."""
+    instructions: str | None
+    """The instructions passed to model."""
 
 
 @dataclass