[FEATURE] Allow users to easily set CachePoints within an Agent's execution

[dbschmigelski]

Problem Statement

Developers currently face cost/latency challenges when building applications that require sequential LLM operations on the same context.

When an agent processes a large amount of data and needs to perform additional operations like structured output formatting, the SDK forces a complete reprocessing of the entire message history. For example, if an agent analyzes 100,000 tokens of content and then needs to generate a structured output, the system must process those same 100,000 tokens again, doubling the token consumption and associated costs.

This makes it difficult or impossible to build cost-effective applications that require both long-running analysis and structured outputs, as there is no built-in way to cache or efficiently reuse the already processed context.

Message caching through CachePoint is already supported. But, this addresses caching only at the start of a message. If an agent generates a large number of tokens during its execution there is no easy way to set cache points within the run. It is technically possible with Hooks, but this is a common scenario that should be made easier by the SDK.

Proposed Solution

No response

Use Case

This is useful when structured output is needed after an Agent generated many tokens during its run.

Alternatives Solutions

No response

Additional Context

No response

[kaleko]

+1 to this feature request. I'd like to be able to do have a setting for agents to add a prompt cache point after each internal message it sends.

This would greatly reduce input tokens from event loops with multiple internal messages and tool calls.

E.g. can we have something like a "cache_messages" argument in the BedrockModel constructor?:

bedrock_model = BedrockModel(model_id, boto_session, cache_prompt="default", cache_tools="default", cache_messages="default")

This would also reduce input tokens down from 2N to N for structured_output use cases like this:

agent("run some big analysis on huge input files")
agent.structured_output(VerySmallStructuredOutputSchema, "use your previous analysis to make my structured output")

[dbschmigelski]

Heavily related to, if not entirely captured by, #756 #404

[kaleko]

Similar common example which unnecessarily re-analyzes the same input tokens multiple times:

Lets say I have tool A which returns 100k tokens and tool B which returns 50 tokens. I instruct my agent to "first use tool A, then write a summary of the response, then use tool B, then write a summary of the response".
My event loop will be system prompt, tool A use, agent message (writing the summary), tool B use, agent message (writing the summary).

In this case, the total input tokens would be > 200k because I've re-read the tool A response twice.