File tree Expand file tree Collapse file tree 1 file changed +19
-4
lines changed
pydantic_ai_slim/pydantic_ai/models Expand file tree Collapse file tree 1 file changed +19
-4
lines changed Original file line number Diff line number Diff line change @@ -802,11 +802,26 @@ def _limit_cache_points(
802802 ) -> None :
803803 """Limit the number of cache points in the request to Anthropic's maximum.
804804
805+ Anthropic enforces a maximum of 4 cache points per request. This method ensures
806+ compliance by counting existing cache points and removing excess ones from messages.
807+
805808 Strategy:
806- 1. Keep the last cache point in system_prompt and tools (if present)
807- 2. Count cache points already used in system_prompt and tools
808- 3. Traverse messages from newest to oldest, keeping the most recent cache points
809- until the maximum limit is reached
809+ 1. Count cache points in system_prompt (can be multiple if list of blocks)
810+ 2. Count cache points in tools (can be in any position, not just last)
811+ 3. Raise UserError if system + tools already exceed MAX_CACHE_POINTS
812+ 4. Calculate remaining budget for message cache points
813+ 5. Traverse messages from newest to oldest, keeping the most recent cache points
814+ within the remaining budget
815+ 6. Remove excess cache points from older messages to stay within limit
816+
817+ Cache point priority (always preserved):
818+ - System prompt cache points
819+ - Tool definition cache points
820+ - Message cache points (newest first, oldest removed if needed)
821+
822+ Raises:
823+ UserError: If system_prompt and tools combined already exceed MAX_CACHE_POINTS (4).
824+ This indicates a configuration error that cannot be auto-fixed.
810825 """
811826 MAX_CACHE_POINTS = 4
812827
You can’t perform that action at this time.
0 commit comments