Include thinking parts in model requests to improve performance and cache hit rates (#2823)

Author: DouweM

docs/thinking.md

@@ -6,27 +6,20 @@ providing its final answer.
 This capability is typically disabled by default and depends on the specific model being used.
 See the sections below for how to enable thinking for each provider.
 
-Internally, if the model doesn't provide thinking objects, Pydantic AI will convert thinking blocks
-(`"<think>..."</think>"`) in provider-specific text parts to `ThinkingPart`s. We have also made
-the decision not to send `ThinkingPart`s back to the provider in multi-turn conversations -
-this helps save costs for users. In the future, we plan to add a setting to customize this behavior.
-
 ## OpenAI
 
-When using the [`OpenAIChatModel`][pydantic_ai.models.openai.OpenAIChatModel], thinking objects are not created
-by default. However, the text content may contain `"<think>"` tags. When this happens, Pydantic AI will
-convert them to [`ThinkingPart`][pydantic_ai.messages.ThinkingPart] objects.
+When using the [`OpenAIChatModel`][pydantic_ai.models.openai.OpenAIChatModel], text output inside `<think>` tags are converted to [`ThinkingPart`][pydantic_ai.messages.ThinkingPart] objects.
+You can customize the tags using the [`thinking_tags`][pydantic_ai.profiles.ModelProfile.thinking_tags] field on the [model profile](models/openai.md#model-profile).
 
-In contrast, the [`OpenAIResponsesModel`][pydantic_ai.models.openai.OpenAIResponsesModel] does
-generate thinking parts. To enable this functionality, you need to set the `openai_reasoning_effort` and
-`openai_reasoning_summary` fields in the
+The [`OpenAIResponsesModel`][pydantic_ai.models.openai.OpenAIResponsesModel] can generate native thinking parts.
+To enable this functionality, you need to set the `openai_reasoning_effort` and `openai_reasoning_summary` fields in the
 [`OpenAIResponsesModelSettings`][pydantic_ai.models.openai.OpenAIResponsesModelSettings].
 
 ```python {title="openai_thinking_part.py"}
 from pydantic_ai import Agent
 from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
 
-model = OpenAIResponsesModel('o3-mini')
+model = OpenAIResponsesModel('gpt-5')
 settings = OpenAIResponsesModelSettings(
     openai_reasoning_effort='low',
     openai_reasoning_summary='detailed',
@@ -37,29 +30,44 @@ agent = Agent(model, model_settings=settings)
 
 ## Anthropic
 
-Unlike other providers, Anthropic includes a signature in the thinking part. This signature is used to
-ensure that the thinking part has not been tampered with. To enable thinking, use the `anthropic_thinking`
-field in the [`AnthropicModelSettings`][pydantic_ai.models.anthropic.AnthropicModelSettings].
+To enable thinking, use the `anthropic_thinking` field in the [`AnthropicModelSettings`][pydantic_ai.models.anthropic.AnthropicModelSettings].
 
 ```python {title="anthropic_thinking_part.py"}
 from pydantic_ai import Agent
 from pydantic_ai.models.anthropic import AnthropicModel, AnthropicModelSettings
 
-model = AnthropicModel('claude-3-7-sonnet-latest')
+model = AnthropicModel('claude-sonnet-4-0')
 settings = AnthropicModelSettings(
     anthropic_thinking={'type': 'enabled', 'budget_tokens': 1024},
 )
 agent = Agent(model, model_settings=settings)
 ...
 ```
 
+## Google
+
+To enable thinking, use the `google_thinking_config` field in the
+[`GoogleModelSettings`][pydantic_ai.models.google.GoogleModelSettings].
+
+```python {title="google_thinking_part.py"}
+from pydantic_ai import Agent
+from pydantic_ai.models.google import GoogleModel, GoogleModelSettings
+
+model = GoogleModel('gemini-2.5-pro')
+settings = GoogleModelSettings(google_thinking_config={'include_thoughts': True})
+agent = Agent(model, model_settings=settings)
+...
+```
+
+## Bedrock
+
 ## Groq
 
 Groq supports different formats to receive thinking parts:
 
-- `"raw"`: The thinking part is included in the text content with the `"<think>"` tag.
+- `"raw"`: The thinking part is included in the text content inside `<think>` tags, which are automatically converted to [`ThinkingPart`][pydantic_ai.messages.ThinkingPart] objects.
 - `"hidden"`: The thinking part is not included in the text content.
-- `"parsed"`: The thinking part has its own [`ThinkingPart`][pydantic_ai.messages.ThinkingPart] object.
+- `"parsed"`: The thinking part has its own structured part in the response which is converted into a [`ThinkingPart`][pydantic_ai.messages.ThinkingPart] object.
 
 To enable thinking, use the `groq_reasoning_format` field in the
 [`GroqModelSettings`][pydantic_ai.models.groq.GroqModelSettings]:
@@ -74,21 +82,15 @@ agent = Agent(model, model_settings=settings)
 ...
 ```
 
-## Google
+## Mistral
 
-To enable thinking, use the `google_thinking_config` field in the
-[`GoogleModelSettings`][pydantic_ai.models.google.GoogleModelSettings].
+Thinking is supported by the `magistral` family of models. It does not need to be specifically enabled.
 
-```python {title="google_thinking_part.py"}
-from pydantic_ai import Agent
-from pydantic_ai.models.google import GoogleModel, GoogleModelSettings
+## Cohere
 
-model = GoogleModel('gemini-2.5-pro-preview-03-25')
-settings = GoogleModelSettings(google_thinking_config={'include_thoughts': True})
-agent = Agent(model, model_settings=settings)
-...
-```
+Thinking is supported by the `command-a-reasoning-08-2025` model. It does not need to be specifically enabled.
 
-## Mistral / Cohere
+## Hugging Face
 
-Neither Mistral nor Cohere generate thinking parts.
+Text output inside `<think>` tags is automatically converted to [`ThinkingPart`][pydantic_ai.messages.ThinkingPart] objects.
+You can customize the tags using the [`thinking_tags`][pydantic_ai.profiles.ModelProfile.thinking_tags] field on the [model profile](models/openai.md#model-profile).

pydantic_ai_slim/pydantic_ai/_parts_manager.py

@@ -156,6 +156,7 @@ def handle_thinking_delta(
         content: str | None = None,
         id: str | None = None,
         signature: str | None = None,
+        provider_name: str | None = None,
     ) -> ModelResponseStreamEvent:
         """Handle incoming thinking content, creating or updating a ThinkingPart in the manager as appropriate.
 
@@ -170,6 +171,7 @@ def handle_thinking_delta(
             content: The thinking content to append to the appropriate ThinkingPart.
             id: An optional id for the thinking part.
             signature: An optional signature for the thinking content.
+            provider_name: An optional provider name for the thinking part.
 
         Returns:
             A `PartStartEvent` if a new part was created, or a `PartDeltaEvent` if an existing part was updated.
@@ -199,24 +201,20 @@ def handle_thinking_delta(
             if content is not None:
                 # There is no existing thinking part that should be updated, so create a new one
                 new_part_index = len(self._parts)
-                part = ThinkingPart(content=content, id=id, signature=signature)
+                part = ThinkingPart(content=content, id=id, signature=signature, provider_name=provider_name)
                 if vendor_part_id is not None:  # pragma: no branch
                     self._vendor_id_to_part_index[vendor_part_id] = new_part_index
                 self._parts.append(part)
                 return PartStartEvent(index=new_part_index, part=part)
             else:
                 raise UnexpectedModelBehavior('Cannot create a ThinkingPart with no content')
         else:
-            if content is not None:
-                # Update the existing ThinkingPart with the new content delta
-                existing_thinking_part, part_index = existing_thinking_part_and_index
-                part_delta = ThinkingPartDelta(content_delta=content)
-                self._parts[part_index] = part_delta.apply(existing_thinking_part)
-                return PartDeltaEvent(index=part_index, delta=part_delta)
-            elif signature is not None:
-                # Update the existing ThinkingPart with the new signature delta
+            if content is not None or signature is not None:
+                # Update the existing ThinkingPart with the new content and/or signature delta
                 existing_thinking_part, part_index = existing_thinking_part_and_index
-                part_delta = ThinkingPartDelta(signature_delta=signature)
+                part_delta = ThinkingPartDelta(
+                    content_delta=content, signature_delta=signature, provider_name=provider_name
+                )
                 self._parts[part_index] = part_delta.apply(existing_thinking_part)
                 return PartDeltaEvent(index=part_index, delta=part_delta)
             else:

pydantic_ai_slim/pydantic_ai/messages.py

@@ -895,7 +895,18 @@ class ThinkingPart:
     signature: str | None = None
     """The signature of the thinking.
 
-    The signature is only available on the Anthropic models.
+    Supported by:
+
+    * Anthropic (corresponds to the `signature` field)
+    * Bedrock (corresponds to the `signature` field)
+    * Google (corresponds to the `thought_signature` field)
+    * OpenAI (corresponds to the `encrypted_content` field)
+    """
+
+    provider_name: str | None = None
+    """The name of the provider that generated the response.
+
+    Signatures are only sent back to the same provider.
     """
 
     part_kind: Literal['thinking'] = 'thinking'
@@ -980,7 +991,10 @@ class BuiltinToolCallPart(BaseToolCallPart):
     _: KW_ONLY
 
     provider_name: str | None = None
-    """The name of the provider that generated the response."""
+    """The name of the provider that generated the response.
+
+    Built-in tool calls are only sent back to the same provider.
+    """
 
     part_kind: Literal['builtin-tool-call'] = 'builtin-tool-call'
     """Part type identifier, this is available on all parts as a discriminator."""
@@ -1198,6 +1212,12 @@ class ThinkingPartDelta:
     Note this is never treated as a delta — it can replace None.
     """
 
+    provider_name: str | None = None
+    """Optional provider name for the thinking part.
+
+    Signatures are only sent back to the same provider.
+    """
+
     part_delta_kind: Literal['thinking'] = 'thinking'
     """Part delta type identifier, used as a discriminator."""
 
@@ -1222,14 +1242,18 @@ def apply(self, part: ModelResponsePart | ThinkingPartDelta) -> ThinkingPart | T
         if isinstance(part, ThinkingPart):
             new_content = part.content + self.content_delta if self.content_delta else part.content
             new_signature = self.signature_delta if self.signature_delta is not None else part.signature
-            return replace(part, content=new_content, signature=new_signature)
+            new_provider_name = self.provider_name if self.provider_name is not None else part.provider_name
+            return replace(part, content=new_content, signature=new_signature, provider_name=new_provider_name)
         elif isinstance(part, ThinkingPartDelta):
             if self.content_delta is None and self.signature_delta is None:
                 raise ValueError('Cannot apply ThinkingPartDelta with no content or signature')
-            if self.signature_delta is not None:
-                return replace(part, signature_delta=self.signature_delta)
             if self.content_delta is not None:
-                return replace(part, content_delta=self.content_delta)
+                part = replace(part, content_delta=(part.content_delta or '') + self.content_delta)
+            if self.signature_delta is not None:
+                part = replace(part, signature_delta=self.signature_delta)
+            if self.provider_name is not None:
+                part = replace(part, provider_name=self.provider_name)
+            return part
         raise ValueError(  # pragma: no cover
             f'Cannot apply ThinkingPartDeltas to non-ThinkingParts or non-ThinkingPartDeltas ({part=}, {self=})'
         )