-
Notifications
You must be signed in to change notification settings - Fork 617
Description
Problem Statement
Currently, BedrockModel has two cache-related options with inconsistent deprecation status:
cache_prompt- already deprecated (warns to useSystemContentBlockwithcachePoint)cache_tools- not yet deprecated (no warning)
With the introduction of CacheConfig(strategy="auto") in PR #1438, there's now a cleaner way to handle prompt caching. However, the cache_tools option remains without a clear migration path or deprecation warning.
This inconsistency may confuse users about which options are recommended.
Proposed Solution
Add a deprecation warning for cache_tools similar to cache_prompt:
if self.config.get("cache_tools"):
warnings.warn(
"cache_tools is deprecated. Use <recommended_alternative> instead.",
UserWarning, stacklevel=3
)Questions before implementation:
- Should
cache_toolsbe deprecated? - What is the recommended alternative?
- Does
CacheConfig(strategy="auto")cover tool caching use cases? - Or should users use
cachePointin the tools list directly?
- Does
Use Case
When users configure BedrockModel with both cache_prompt and cache_tools, only cache_prompt shows a deprecation warning. This creates confusion about whether cache_tools is still a recommended option.
model = BedrockModel(
model_id="us.anthropic.claude-sonnet-4-5-20250929-v1:0",
cache_prompt="default", # ⚠️ Warning shown
cache_tools="default", # No warning - is this still OK to use?
)Consistent deprecation messaging would help users migrate to the recommended approach.
Alternatives Solutions
-
Keep
cache_toolsas-is - If there's no suitable alternative yet, document thatcache_toolsis the only way to cache tool definitions. -
Extend
CacheConfigto support tools - Add acache_toolsstrategy option toCacheConfigfor unified caching configuration.
Additional Context
- Related: feat(bedrock): add automatic prompt caching support #1438 (automatic prompt caching with
CacheConfig) - Related: [FEATURE] Prompt caching support for all models #1140 (prompt caching support for all models)
- Related: [FEATURE] cache_messages argument for Bedrock model provider #404 (cache_messages feature request)
- Related: [FEATURE] Amazon Nova tool caching with cachePoint in messages #449 (Amazon Nova tool caching)
I have a draft implementation ready and can submit a PR once the migration path is clarified.