Skip to content

Conversation

@dgageot
Copy link
Member

@dgageot dgageot commented Jan 15, 2026

  • Cut the message that contained the date, env, agents.md and skills into separate system messages
  • Reordered the system messages to have the most invariant messages first
  • Instead of caching the first two system messages and the last two user/assistant messages, we mark the tool instructions with a cache control checkpoint because it's the last system message before the ones that can be more user specific
  • We mark the last system message too.
  • We keep on marking the last user/assistant messages with the cache control flag.

Added a few tests and e2e on the way.

@dgageot dgageot requested a review from a team as a code owner January 15, 2026 16:24
@dgageot
Copy link
Member Author

dgageot commented Jan 15, 2026

/review

Copy link
Contributor

@simonferquel simonferquel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small changes to make things a tiny bit more robust

}
}

// Cache control checkpoint #1 out of 4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit brittle to only rely on where you add messages in the « system slices » compared to that line of code to make it part of what we think is invariant or not. It is a bit too easy to break the cache.
Can we separate the messages into different slices (invariant messages, lowFrequencyChangeMessages, …) and put the checkpoint when we merge them (in correct order) in the system messages slice?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@simonferquel Can we merge this and you prepare another PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

@simonferquel simonferquel self-requested a review January 16, 2026 10:01
Copy link
Contributor

@simonferquel simonferquel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unblocking, fixing in followup

@simonferquel simonferquel merged commit 7772759 into docker:main Jan 16, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants