Add Context Synthesization Feature #3017

shariqriazz · 2025-04-28T23:27:56Z

Overview

This feature introduces intelligent context management through synthesization, significantly enhancing Roo Code's ability to maintain conversation coherence during extended interactions. Unlike the existing sliding window approach that simply truncates conversations, context synthesization uses AI to create concise summaries of conversation history, preserving critical information while reducing token usage.

Key Features

Automatic Context Synthesization

Smart Threshold Detection: Automatically triggers synthesization when context window usage approaches a configurable threshold
Configurable Preservation: Maintains specified numbers of initial and recent conversation turns in full detail
Middle Section Compression: Intelligently synthesizes the middle portion of conversations while preserving critical context

Manual Context Synthesization

On-Demand Compression: Allows users to trigger synthesization manually via context menu
Visual Feedback: Provides clear status indicators during the synthesization process
Seamless Continuation: Enables uninterrupted conversation flow after synthesization

User Configuration

Enable/Disable: Toggle automatic synthesization on/off
Threshold Control: Set the percentage of context window that triggers automatic synthesization
Preservation Settings: Configure how many initial and recent messages to keep intact

Comparison with Sliding Window

Feature	Sliding Window	Context Synthesization
Approach	Simple truncation by removing messages	AI-powered summarization of conversation history
Information Retention	Loses information from removed messages	Preserves key information in compressed form
Preservation Strategy	Keeps first message and removes a fraction of older messages	Preserves configurable numbers of initial and recent messages
Context Continuity	Can create context gaps	Maintains conversation coherence
Token Efficiency	Reduces tokens by removing content	Reduces tokens while maximizing information density
User Control	Limited configuration	Extensive configuration options
Manual Triggering	Not available	Available through context menu
Visual Feedback	None	Status indicator shows progress and completion

Technical Implementation

Core Components

ContextSynthesizer Service
- Handles the AI-powered synthesis of conversation segments
- Uses a specialized prompt optimized for technical context preservation
- Prioritizes code context, tool usage, task progress, and technical information
Integration with Conversation Flow
- Monitors context window usage
- Applies configurable rules for when to synthesize
- Seamlessly integrates synthesized content into the conversation
UI Components
- SynthesizingIndicator for visual feedback
- Context menu integration for manual triggering
- Settings panel for configuration

Key Differences from Sliding Window

The sliding window implementation (src/core/sliding-window/index.ts) uses a simple truncation approach:

// Sliding Window approach
export function truncateConversation(
  messages: Anthropic.Messages.MessageParam[],
  fracToRemove: number,
): Anthropic.Messages.MessageParam[] {
  const truncatedMessages = [messages[0]]
  const rawMessagesToRemove = Math.floor((messages.length - 1) * fracToRemove)
  const messagesToRemove = rawMessagesToRemove - (rawMessagesToRemove % 2)
  const remainingMessages = messages.slice(messagesToRemove + 1)
  truncatedMessages.push(...remainingMessages)

  return truncatedMessages
}

In contrast, the new synthesization approach (src/services/synthesization/ContextSynthesizer.ts) uses AI to create an intelligent summary:

async synthesize(messagesToSynthesize: Anthropic.MessageParam[]): Promise<Anthropic.MessageParam | null> {
  // ...
  // Use AI to create a dense, information-rich synthesis
  const stream = this.apiHandler.createMessage(
    systemPrompt,
    [{ role: "user", content: userPrompt }],
    undefined
  )
  
  // Process the response
  // ...
  
  // Return the synthesis as a user message
  return {
    role: "user",
    content: `[Synthesized Conversation History]\n${synthesisText.trim()}`,
  }
}

Benefits

Enhanced Context Management: Preserves critical information that would be lost with simple truncation
Improved Conversation Coherence: Maintains continuity between early and recent interactions
Token Efficiency: Maximizes information density while minimizing token usage
User Control: Provides both automatic and manual options with extensive configuration
Technical Focus: Optimized for preserving code context and technical details

Use Cases

Long Debugging Sessions: Maintain context about errors and solutions throughout extended debugging
Complex Feature Development: Preserve architectural decisions and implementation details
Multi-Stage Projects: Keep track of progress across different implementation phases
Reference-Heavy Work: Retain information about files, code structures, and their relationships

Future Enhancements

Dedicated model selection for synthesization to optimize for speed/cost
Enhanced synthesis quality metrics and feedback mechanisms
Integration with other context management features
Preservation of specific message types based on importance

Important

This pull request adds a context synthesization feature to summarize older conversation turns, with both automatic and manual triggers, configurable settings, and integration into the conversation flow and UI components.

Behavior:
- Adds context synthesization feature to summarize older conversation turns, preserving key information while reducing token usage.
- Supports both automatic and manual triggers for synthesization.
- Configurable settings for enabling/disabling, threshold control, and preservation settings.
Core Components:
- Introduces ContextSynthesizer service in src/services/synthesization/ContextSynthesizer.ts for AI-powered synthesis.
- Updates Cline.ts to integrate synthesization with conversation flow.
- Adds UI components like SynthesizingIndicator and context menu options for manual triggering.
Misc:
- Updates ClineProvider.ts and webviewMessageHandler.ts to handle new settings and manual synthesization messages.
- Adds tests in ClineProvider.test.ts for context summarization settings.
- Updates i18n files for new settings descriptions.

^{This description was created by}^{for 115efcc. You can customize this summary. It will automatically update as commits are pushed.}

Implements the optional context summarization feature based on the plan in (now deleted) `context-summarization-plan.md`. Key changes: - Adds new settings (`enableContextSummarization`, thresholds, turn counts) to control the feature, accessible in Settings -> Context Management. Settings are persistent and included in import/export. - Creates `ContextSummarizer` service to handle summarization logic, using the main configured LLM with a refined, directive prompt. - Modifies `Cline.ts` to conditionally call the summarizer or fallback to truncation based on settings and token thresholds before API requests. - Adds a system message `[Older conversation turns summarized...]` to the chat UI for transparency when summarization occurs. - Includes necessary updates to schemas, types, message handlers, webview context, and tests. - Fixes UI state bugs related to settings persistence and interaction.

changeset-bot · 2025-04-28T23:28:00Z

⚠️ No Changeset found

Latest commit: 2c3064d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

ellipsis-dev · 2025-04-28T23:30:34Z

The pull request is quite large, with 39 changed files and 1239 lines added. While the changes primarily focus on adding context summarization features, there are also updates to settings and tests. It might be beneficial to split the pull request into smaller, more manageable parts. Consider separating the context summarization feature implementation from the settings updates and tests. This will make it easier to review and ensure each part is thoroughly tested and validated.

ellipsis-dev · 2025-04-28T23:31:11Z

src/core/webview/webviewMessageHandler.ts

 			await updateGlobalState("historyPreviewCollapsed", message.bool ?? false)
 			// No need to call postStateToWebview here as the UI already updated optimistically
 			break
+		// Context Synthesization Settings


There is an inconsistent use of terminology for the context feature. The comments mention 'Context Synthesization Settings', but the state keys and message cases use 'Context Summarization' (e.g., 'enableContextSummarization', 'manualSynthesize'). Consider standardizing the terminology across the code for clarity.

Suggested change

// Context Synthesization Settings

// Context Summarization Settings

^{This comment was generated because it violated a code review rule: mrule_aQsEnH8jWdOfHq2Z.}

We do need to unify terminology

ellipsis-dev · 2025-04-28T23:31:11Z

src/shared/WebviewMessage.ts

 		| "searchFiles"
 		| "toggleApiConfigPin"
 		| "setHistoryPreviewCollapsed"
+		// Context Synthesizing Settings


There is an inconsistency in the naming conventions for the context synthesization/summarization settings. The comment above the new union values says "Context Synthesizing Settings" while the union type options use "ContextSummarization..." (e.g., enableContextSummarization, contextSummarizationTriggerThreshold) along with "manualSynthesize" and "synthesizationStatus". Consider using a consistent term (either 'synthesization' or 'summarization') for clarity and to avoid confusion.

Suggested change

// Context Synthesizing Settings

// Context Summarization Settings

^{This comment was generated because it violated a code review rule: mrule_aQsEnH8jWdOfHq2Z.}

ellipsis-dev · 2025-04-28T23:31:11Z

webview-ui/src/components/settings/ContextManagementSettings.tsx

+						onChange={(e: any) => setCachedStateField("enableContextSummarization", e.target.checked)} // Use generic setter
+						data-testid="enable-context-summarization-checkbox">
+						<label className="block font-medium mb-1">
+							{t("settings:contextManagement.synthesization.enable.label")}


There is a naming inconsistency: the component and props refer to 'Context Summarization' while the translation keys use 'synthesization' (e.g. t("settings:contextManagement.synthesization.enable.label")). It would be clearer if a single term was chosen for consistency throughout the codebase.

^{This comment was generated because it violated a code review rule: mrule_aQsEnH8jWdOfHq2Z.}

ellipsis-dev · 2025-04-28T23:31:11Z

webview-ui/src/context/ExtensionStateContext.tsx

 	mcpServers: McpServer[]
 	hasSystemPromptOverride?: boolean
 	currentCheckpoint?: string
+	synthesizationStatus?: {


There is an inconsistency in naming: the state property is named synthesizationStatus (with an 'i') while the default settings and setters use ...Summarization... (with an 'a'). Please standardize the terminology (either use 'synthesization' or 'summarization') to avoid confusion.

^{This comment was generated because it violated a code review rule: mrule_aQsEnH8jWdOfHq2Z.}

ellipsis-dev · 2025-04-28T23:31:11Z

webview-ui/src/i18n/locales/ca/settings.json

+				"label": "Torns recents a mantenir",
+				"description": "Nombre dels missatges de conversa més recents a mantenir sempre amb detall complet."
+			},
+			"turns": "missatges"


Minor lexicographical inconsistency: In the 'summarization' block, the key 'turns' is labeled as "missatges", which contrasts with the use of "Torns" in the 'initialTurns' and 'recentTurns' keys. Consider using a consistent term (e.g., "torns") for conversation turns to avoid confusion.

Suggested change

"turns": "missatges"

"turns": "torns"

adamhill · 2025-04-28T23:43:02Z

Nice. My Venmo is open for naming rights :-)

shariqriazz · 2025-04-28T23:44:31Z

Nice. My Venmo is open for naming rights :-)

haha! the cost of rebranding ..! i think i need to fix some tests too

shariqriazz · 2025-05-05T20:06:04Z

This needs to have another update, currently disabling auto synthesize stops the manual @ mention also.

…etting

shariqriazz · 2025-05-05T23:09:38Z

its fixed now.

cte · 2025-05-06T23:52:23Z

@shariqriazz - Thanks so much for getting the ball rolling on this. We're discussing internally what the strategy should be for this class of feature (also related: #2971). We're going to figure it out for next week's release.

shariqriazz · 2025-05-06T23:58:57Z

@cte Perfect, looking forward for this

canrobins13 · 2025-05-16T16:42:57Z

Thanks for the inspiration here! We added some version of this in #3582, which you can enable with the autoCondenseContext experimental feature.
Some follow up features are tracked in these issues:

hannesrudolph · 2025-05-16T18:22:58Z

@shariqriazz The primary concern with your PR was the large surface area and complexity of the changes relative to what the feature actually delivered. Your implementation introduced a significant amount of risk and potential technical debt. Another PR was merged because it achieved the same functionality but with a smaller, more manageable footprint and less complexity. Sorry for not working more closely with you on this earlier on to help get this across the finish line. We really do appreciate everything you done on this.

shariqriazz added 11 commits April 27, 2025 13:33

types

9419866

feat(i18n): add missing context summarization translations

a199005

rm

7fc2b77

Merge branch 'origin/main' into context-management

ad6b7b9

Add context summarization feature with manual trigger option

08adf5f

Improve manual summarization UX with visual feedback

67b1ac5

Move Summarize option to the top of context menu

e7f1cbe

@mention-buggy

14c85de

manual-fixed

c15680c

sythesize

115efcc

shariqriazz requested review from cte and mrubens as code owners April 28, 2025 23:27

github-project-automation bot added this to Roo Code Roadmap Apr 28, 2025

github-project-automation bot moved this to New in Roo Code Roadmap Apr 28, 2025

dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. enhancement New feature or request labels Apr 28, 2025

ellipsis-dev bot reviewed Apr 28, 2025

View reviewed changes

fix-cline-conflict

d9a1ef6

cte and others added 3 commits April 28, 2025 19:55

Merge branch 'main' into context-management

de366cb

fix-inconsistencies

13d239c

i18n: Add missing backend context synthesization translations

6819e0d

dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Apr 29, 2025

shariqriazz added 2 commits April 29, 2025 14:36

fix(test): Update context mention and settings tests

7448edb

Merge branch 'main' into context-management

445d411

hannesrudolph moved this from New to PR [Pre Approval Review] in Roo Code Roadmap Apr 29, 2025

mrubens assigned cte May 5, 2025

hannesrudolph moved this from PR [Pre Approval Review] to PR [Greenlit] in Roo Code Roadmap May 5, 2025

shariqriazz added 2 commits May 6, 2025 03:53

fix: Allow manual synthesize trigger independent of auto-synthesize s…

2a74867

…etting

feat: Update default synthesizer turns to 3 initial, 3 recent

2c3064d

hannesrudolph closed this May 16, 2025

github-project-automation bot moved this from PR [Greenlit] to Done in Roo Code Roadmap May 16, 2025

mrubens added this to Roo Code Roadmap May 20, 2025

github-project-automation bot moved this to New in Roo Code Roadmap May 20, 2025

hannesrudolph moved this from New to Done in Roo Code Roadmap May 20, 2025

shariqriazz deleted the context-management branch June 21, 2025 18:26

	// Context Synthesization Settings
	// Context Summarization Settings

	// Context Synthesizing Settings
	// Context Summarization Settings

Add Context Synthesization Feature #3017

Add Context Synthesization Feature #3017

Uh oh!

Conversation

shariqriazz commented Apr 28, 2025 • edited by ellipsis-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Key Features

Automatic Context Synthesization

Manual Context Synthesization

User Configuration

Comparison with Sliding Window

Technical Implementation

Core Components

Key Differences from Sliding Window

Benefits

Use Cases

Future Enhancements

Uh oh!

changeset-bot bot commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

ellipsis-dev bot commented Apr 28, 2025

Uh oh!

ellipsis-dev bot Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

adamhill Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

adamhill Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

adamhill Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

adamhill Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

adamhill commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shariqriazz commented Apr 28, 2025

Uh oh!

shariqriazz commented May 5, 2025

Uh oh!

shariqriazz commented May 5, 2025

Uh oh!

cte commented May 6, 2025

Uh oh!

shariqriazz commented May 6, 2025

Uh oh!

canrobins13 commented May 16, 2025

Uh oh!

hannesrudolph commented May 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

shariqriazz commented Apr 28, 2025 •

edited by ellipsis-dev bot

Loading

changeset-bot bot commented Apr 28, 2025 •

edited

Loading

adamhill commented Apr 28, 2025 •

edited

Loading