fix: prevent context condensing from triggering too early in new conversations #8159
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ersations - Added MIN_TOKENS_FOR_PERCENTAGE_TRIGGER constant (1000 tokens) - Modified percentage-based condensing logic to require minimum token threshold - Prevents condensing on brand new conversations with low percentage thresholds - Added comprehensive tests for the new behavior Fixes #8158
dosubot
bot
added
size:L
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
| * Minimum number of tokens required before percentage-based condensing can trigger. | ||
| * This prevents condensing from happening too early in new conversations. | ||
| */ | ||
| export const MIN_TOKENS_FOR_PERCENTAGE_TRIGGER = 1000 |
Contributor
Author
There was a problem hiding this comment.
Could we consider making this threshold configurable in the future? While 1000 tokens works well as a default, some users might benefit from being able to adjust this based on their specific use cases.
| // Only trigger percentage-based condensing if we have enough context | ||
| // This prevents condensing from happening on brand new conversations | ||
| const shouldCondenseByPercentage = | ||
| contextPercent >= effectiveThreshold && prevContextTokens >= MIN_TOKENS_FOR_PERCENTAGE_TRIGGER |
Contributor
Author
There was a problem hiding this comment.
Would it be helpful to add a debug log here when condensing is skipped due to the minimum threshold? Something like:
This could help with debugging and understanding why condensing isn't triggering in certain scenarios.
| currentProfileId: "default", | ||
| }) | ||
|
|
||
| // Verify summarizeConversation was not called even though percentage (0.9%) would exceed threshold (5%) |
Contributor
Author
There was a problem hiding this comment.
Minor nit: For consistency with other test comments, could this be "(0.9% of context window)" instead of just "(0.9%)"?
hannesrudolph
added
the
Issue/PR - Triage
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR attempts to address Issue #8158. Feedback and guidance are welcome.
Problem
Intelligent context condensing was triggering too early in brand new conversations when users had low percentage thresholds configured (e.g., 5%), causing a loop of "context condensed" messages immediately after the first user message.
Solution
Added a minimum token threshold (
MIN_TOKENS_FOR_PERCENTAGE_TRIGGER = 1000) that must be met before percentage-based condensing can trigger. This prevents condensing from happening on brand new conversations while maintaining the feature's effectiveness for longer conversations.Changes
MIN_TOKENS_FOR_PERCENTAGE_TRIGGERconstant (1000 tokens) insrc/core/sliding-window/index.tsTesting
Fixes #8158
Important
Introduces a minimum token threshold to prevent premature context condensing in new conversations, addressing Issue #8158.
MIN_TOKENS_FOR_PERCENTAGE_TRIGGER(1000 tokens) inindex.tsto prevent early context condensing in new conversations.truncateConversationIfNeeded()to require both context percentage and token count thresholds.sliding-window.spec.tsto verify condensing does not trigger below the token threshold and works correctly above it.This description was created by
for 395640c. You can customize this summary. It will automatically update as commits are pushed.