Prompt caching/conversation summarization

Author: roblourens

release-notes/images/1_100/summarized-conversation.png

@@ -97,7 +97,19 @@ The keyboard shortcut `kb(workbench.action.chat.open)` still just opens the chat
 
 ### Agent informed of undos and manual edits
 
-Previously, making manual edits during an Agent mode session could confuse the model. Now, the agent is prompted about your changes, and should re-read files when necessary before editing files that may have changed. 
+Previously, making manual edits during an agent mode session could confuse the model. Now, the agent is prompted about your changes, and should re-read files when necessary before editing files that may have changed.
+
+### Conversation history summarized and optimized for prompt caching
+
+We've made some changes to how our agent mode prompt is built to optimize for prompt caching. Prompt caching is a way to speed up model responses by maintaining a stable prefix for the prompt. The next request is able to resume from that prefix, and the result is that each request should be a bit faster. This is especially effective in a repetitive series of requests with large context, like you typically have in agent mode.
+
+When your conversation gets long, or your context gets very large, you may see this message in your agent mode session:
+
+![summarized conversation message](images/1_100/summarized-conversation.png)
+
+Instead of keeping the whole conversation as a FIFO, breaking the cache, we compress the conversation so far into a summary of the most important information and the current state of your task. This keeps the prompt prefix stable, and your responses fast.
+
+If you prefer, you can disable this with `setting(github.copilot.chat.advanced.summarizeAgentConversationHistory)`.
 
 ## Workbench