You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add anthropic_cache_tools and anthropic_cache_instructions settings
This commit addresses maintainer feedback on the Anthropic prompt caching PR:
- Add anthropic_cache_tools field to cache last tool definition
- Add anthropic_cache_instructions field to cache system prompts
- Rewrite existing CachePoint tests to use snapshot() assertions
- Add comprehensive tests for new caching settings
- Remove standalone example file, add docs section instead
- Move imports to top of test files
- Remove ineffective Google CachePoint test
- Add "Supported by: Anthropic" to CachePoint docstring
- Add Anthropic docs link in cache_control method
Tests are written but snapshots not yet generated (will be done in next commit).
Copy file name to clipboardExpand all lines: docs/models/anthropic.md
+108Lines changed: 108 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -77,3 +77,111 @@ model = AnthropicModel(
77
77
agent = Agent(model)
78
78
...
79
79
```
80
+
81
+
## Prompt Caching
82
+
83
+
Anthropic supports [prompt caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) to reduce costs by caching parts of your prompts. PydanticAI provides three ways to use prompt caching:
84
+
85
+
### 1. Cache User Messages with `CachePoint`
86
+
87
+
Insert a [`CachePoint`][pydantic_ai.messages.CachePoint] marker in your user messages to cache everything before it:
88
+
89
+
```python
90
+
from pydantic_ai import Agent, CachePoint
91
+
92
+
agent = Agent('anthropic:claude-sonnet-4-5')
93
+
94
+
# Everything before CachePoint will be cached
95
+
result =await agent.run([
96
+
"Long context that should be cached...",
97
+
CachePoint(),
98
+
"Your question here"
99
+
])
100
+
```
101
+
102
+
### 2. Cache System Instructions
103
+
104
+
Use `anthropic_cache_instructions=True` to cache your system prompt:
105
+
106
+
```python
107
+
from pydantic_ai import Agent
108
+
from pydantic_ai.models.anthropic import AnthropicModelSettings
109
+
110
+
agent = Agent(
111
+
'anthropic:claude-sonnet-4-5',
112
+
system_prompt='Long detailed instructions...',
113
+
model_settings=AnthropicModelSettings(
114
+
anthropic_cache_instructions=True
115
+
),
116
+
)
117
+
118
+
result =await agent.run("Your question")
119
+
```
120
+
121
+
### 3. Cache Tool Definitions
122
+
123
+
Use `anthropic_cache_tools=True` to cache your tool definitions:
124
+
125
+
```python
126
+
from pydantic_ai import Agent
127
+
from pydantic_ai.models.anthropic import AnthropicModelSettings
128
+
129
+
agent = Agent(
130
+
'anthropic:claude-sonnet-4-5',
131
+
model_settings=AnthropicModelSettings(
132
+
anthropic_cache_tools=True
133
+
),
134
+
)
135
+
136
+
@agent.tool
137
+
defmy_tool() -> str:
138
+
"""Tool definition will be cached."""
139
+
return"result"
140
+
141
+
result =await agent.run("Use the tool")
142
+
```
143
+
144
+
### Combining Cache Strategies
145
+
146
+
You can combine all three caching strategies for maximum savings:
147
+
148
+
```python
149
+
from pydantic_ai import Agent, CachePoint
150
+
from pydantic_ai.models.anthropic import AnthropicModelSettings
151
+
152
+
agent = Agent(
153
+
'anthropic:claude-sonnet-4-5',
154
+
system_prompt='Detailed instructions...',
155
+
model_settings=AnthropicModelSettings(
156
+
anthropic_cache_instructions=True,
157
+
anthropic_cache_tools=True,
158
+
),
159
+
)
160
+
161
+
@agent.tool
162
+
defsearch_docs(query: str) -> str:
163
+
"""Search documentation."""
164
+
returnf"Results for {query}"
165
+
166
+
# First call - writes to cache
167
+
result1 =await agent.run([
168
+
"Long context from documentation...",
169
+
CachePoint(),
170
+
"First question"
171
+
])
172
+
173
+
# Subsequent calls - read from cache (90% cost reduction)
174
+
result2 =await agent.run([
175
+
"Long context from documentation...", # Same content
176
+
CachePoint(),
177
+
"Second question"
178
+
])
179
+
```
180
+
181
+
Access cache usage statistics via `result.usage()`:
0 commit comments