feat: add tool result compression to prevent context window exhaustion #6466
Conversation
- Add compressToolResult utility with intelligent truncation - Integrate compression into pushToolResult function - Scale compression limits based on model context window - Add comprehensive tests for compression functionality - Fixes #6463: Tool results that exceed context limits now get compressed
dosubot
bot
added
size:L
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
| @@ -0,0 +1,114 @@ | |||
| import { truncateOutput } from "../../integrations/misc/extract-text" | |||
| import { DEFAULT_TERMINAL_OUTPUT_CHARACTER_LIMIT } from "@roo-code/types" | |||
There was a problem hiding this comment.
This import is unused and should be removed. The constant isn't referenced anywhere in this file, creating unnecessary coupling to the types package.
|
|
||
| if (typeof content === "string") { | ||
| cline.userMessageContent.push({ type: "text", text: content || "(tool did not return anything)" }) | ||
| // Get compression limits based on the model's context window |
There was a problem hiding this comment.
This code block (lines 253-256) is duplicated below at lines 269-271. Could we extract this into a helper function to follow DRY principles? Something like getCompressionLimitsForModel would eliminate the duplication.
| const compressedLines = compressedResult.split("\n").length | ||
|
|
||
| // Add compression info at the beginning to make it clear to the model | ||
| const compressionNote = `[Tool result compressed: Original ${originalLength} characters, ${originalLines} lines → Compressed to ${compressedLength} characters, ${compressedLines} lines to prevent context window exhaustion]\n\n` |
There was a problem hiding this comment.
The compression note format uses arrow symbol and specific wording that differs from the existing truncateOutput function's format. Should we align these for consistency across the codebase?
| } { | ||
| // Conservative approach: tool results should not exceed a small percentage of context window | ||
| // Assuming roughly 4 characters per token on average | ||
| const maxToolResultTokens = Math.floor(contextWindow * 0.1) // 10% of context window |
There was a problem hiding this comment.
Missing edge case handling - what happens if contextWindow <= 0 or NaN? This could cause unexpected behavior with Math.floor() and Math.min() operations.
| // Conservative approach: tool results should not exceed a small percentage of context window | ||
| // Assuming roughly 4 characters per token on average | ||
| const maxToolResultTokens = Math.floor(contextWindow * 0.1) // 10% of context window | ||
| const characterLimit = Math.min(maxToolResultTokens * 4, DEFAULT_TOOL_RESULT_CHARACTER_LIMIT * 2) // Cap at 2x default |
There was a problem hiding this comment.
Consider using named constants for these magic numbers. Something like TOOL_RESULT_CONTEXT_PERCENTAGE = 0.1 and COMPRESSION_CAP_MULTIPLIER = 2 would make the logic clearer.
| const characterLimit = Math.min(maxToolResultTokens * 4, DEFAULT_TOOL_RESULT_CHARACTER_LIMIT * 2) // Cap at 2x default | ||
|
|
||
| // Line limit scales with character limit | ||
| const lineLimit = Math.floor(characterLimit / 50) // Assume ~50 chars per line on average |
There was a problem hiding this comment.
The assumption of ~50 characters per line might not hold for all content types. Consider making this configurable or using a more robust calculation based on actual content analysis.
hannesrudolph
added
the
Issue/PR - Triage
hannesrudolph
added
PR - Needs Preliminary Review
and removed
Issue/PR - Triage
|
The issue needs better scoping, I'll close this but leave the issue open for discussion on the best solution to this problem |
github-project-automation
bot
moved this from PR [Needs Prelim Review]
to Done
in Roo Code Roadmap
Fixes #6463
Summary
This PR implements automatic compression of tool results that exceed context window limits to prevent tasks from being unable to continue when tools return large amounts of data.
Changes
New compression utility (
src/core/tools/compressToolResult.ts):truncateOutputfunctionIntegration with tool result processing (
src/core/assistant-message/presentAssistantMessage.ts):pushToolResultfunctionComprehensive test coverage (
src/core/tools/__tests__/compressToolResult.spec.ts):Technical Details
Testing
Impact
This change ensures that tools returning large results (like search operations) no longer cause tasks to fail due to context window exhaustion, while maintaining the most important parts of the tool output for the model to work with.
Important
Adds tool result compression to prevent context window exhaustion, with dynamic limits and comprehensive testing.
compressToolResult.ts):compressToolResult()for compressing tool results exceeding limits.getCompressionLimitsForContextWindow()to determine limits based on context window size.truncateOutputfor intelligent truncation.presentAssistantMessage.ts):pushToolResult()for string and array tool results.compressToolResult.spec.ts):This description was created by
for ee6d6db. You can customize this summary. It will automatically update as commits are pushed.