Prompt Caching for Claude model

[MarcoLooy]

Describe the feature

Prompt Caching

Prompt Caching is a powerful feature that optimizes your API usage by allowing resuming from specific prefixes in your prompts. This approach significantly reduces processing time and costs for repetitive tasks or prompts with consistent elements.

• https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#can-i-use-prompt-caching-at-the-same-time-as-other-betas

Use Case

Cache prompt

Proposed Solution

https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#can-i-use-prompt-caching-at-the-same-time-as-other-betas

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

latest

Environment details (OS name and version, etc.)

Ubuntu 22

[zshzbh]

Hey @MarcoLooy ,

Thanks for the feedback! Just want to clarify the issue here - do you want to add Prompt Caching feature to AWS SDK JS?

Thanks!
Maggie

[github-actions]

This issue has not received a response in 1 week. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.

[MarcoLooy]

How can we do prompt caching in AWS Bedrock?

[zshzbh]

Hey @MarcoLooy ,

Prompt Caching is a beta Anthropic feature. Bedrock does not adopt features that are in beta. There is currently no official ETA for bedrock support of Anthropic Prompt Cacheing. But I will add add your suggestion to the feature request.

Thanks!
Maggie