-
Notifications
You must be signed in to change notification settings - Fork 2.6k
fix: Context Overload in Read File Tool #6319
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
cte
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think in general the strategy is good, but I left a few questions / comments. Separately, I'm going to test this in a few scenarios just to get a sense of how it behaves.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Adding a dynamic hard cap based on the context window would be a solid improvement for the future, but this is a good first step.
mrubens
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would really like to find a solution that doesn’t require special casing of minified files. Can we try reading lines in chunks instead of reading to the newline every time?
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
resolved! |
|
Is there a plan to merge this soon? |
|
Stale. Implementation not the one we want. |
Fix Context Overload in Read File Tool
Problem
The read_file tool could attempt to read files that exceed the available context window, causing API failures and poor user experience.
Solution
Added preemptive context validation that dynamically calculates available space and applies intelligent file size limits.
Changes
New contextValidator module:
Enhanced readFileTool:
Fix
Turn off partial reads, and open Roo-Code-Evals/javascript/lodash. Ask the following prompt from this task: https://app.roocode.com/share/fa7affcc-611d-46ff-89a4-d741d8593b2a
Remaining concerns:
Fixes #7744
Important
Introduces context validation in the Read File Tool to prevent context overload by dynamically calculating available space and applying file size limits.
contextValidatormodule to calculate available context space and apply file size limits.readFileToolto validate file size before reading and provide instructions for truncated files.contextValidator.test.tsto test context validation logic.readFileTool.spec.tsto test file reading with context validation.read-partial-content.spec.tsto test partial content reading.providerFormat.spec.tsto test provider format logic.api.tsto include provider format mapping logic.This description was created by
for 2285015. You can customize this summary. It will automatically update as commits are pushed.