feat: Enhance read_file tool to support multiple files via setting

[shariqriazz]

Context

This PR enhances the existing read_file tool to allow the AI model to request the content of multiple files within a single operation. This approach was chosen based on feedback to avoid introducing a new experimental tool and instead leverage the existing read_file functionality, controlled by a user setting.

The goal is to improve efficiency and reduce the number of tool calls required for tasks needing context from several sources, while giving users control over the maximum number of files read concurrently.

Implementation

The implementation involved the following key steps:

1. New Setting (maxConcurrentFileReads):

   • Added a new setting maxConcurrentFileReads to src/schemas/index.ts with a default value of 1. This setting controls the maximum number of files the read_file tool can process in a single request when given multiple paths.
   • Updated relevant type definitions (RooCodeSettings, ExtensionState, etc.) in src/schemas/index.ts, src/shared/ExtensionMessage.ts, src/shared/WebviewMessage.ts, and src/exports/roo-code.d.ts.
   • Added UI elements (label, description, slider) for the new setting in the "Context Management" section of the settings view (webview-ui/src/components/settings/ContextManagementSettings.tsx, webview-ui/src/components/settings/SettingsView.tsx, webview-ui/src/i18n/locales/en/settings.json).
   • Updated state management (webview-ui/src/context/ExtensionStateContext.tsx) and persistence logic (src/core/config/ContextProxy.ts, src/core/webview/ClineProvider.ts, src/core/webview/webviewMessageHandler.ts) to handle the new setting.

2. readFileTool Enhancement (src/core/tools/readFileTool.ts):

   • Modified the tool logic to parse the <path> parameter. It now detects if the parameter value is a JSON string array or a single path string.
   • When a JSON array is detected, the tool iterates through the provided paths.
   • Added validation to enforce the maxConcurrentFileReads setting limit. If the number of requested paths exceeds the limit, a specific error is returned.
   • The tool now aggregates results for each file (either <file_content> or <file_error>) within a parent <read_result> tag.
   • Error handling for individual files (non-existent, ignored, read errors) is maintained within the loop, allowing the tool to process other valid files in the request.
   • The start_line and end_line parameters are now explicitly disallowed when multiple files are requested.

3. Tool Prompt Update (src/core/prompts/tools/read-file.ts):

   • Updated the read_file tool description provided to the AI model. It now explains the ability to use a JSON string array for the path parameter, mentions the maxConcurrentFileReads setting, and clarifies that start_line/end_line only apply to single-file reads.
   • Updated examples to show multi-file usage.

4. Testing:

   • Updated existing tests (src/core/__tests__/read-file-tool.test.ts, src/core/__tests__/read-file-xml.test.ts) to cover the new multi-file functionality, including JSON parsing, limit enforcement, error handling, and result aggregation.

How to Test

1. Build and run the extension from the feature/multi-file-read-setting branch.
2. Go to Roo Code Settings -> Context Management.
3. Adjust the "Max Concurrent File Reads" slider (default is 1). Set it to a value greater than 1 (e.g., 3 or 5) and save settings.
4. Start a new chat task.
5. Ask Roo to read multiple files, ensuring the number is less than or equal to the limit set in step 3 (e.g., "Read package.json, README.md, and .gitignore").
6. Verify: Roo should use the <read_file> tool with a JSON array string in the <path> parameter (e.g., <path>["package.json", "README.md", ".gitignore"]</path>).
7. Verify: The tool result (<read_result>) should contain <file_content> tags for successfully read files and <file_error> tags for any files that couldn't be read.
8. Ask Roo to read more files than the limit set in step 3 (e.g., if the limit is 3, ask for 4 files).
9. Verify: Roo should receive a <tool_error> from read_file indicating that the requested number of files exceeds the configured maximum.
10. Ask Roo to read a single file using start_line or end_line.
11. Verify: This should work as before.
12. Ask Roo to read multiple files and include start_line or end_line.
13. Verify: Roo should receive a <tool_error> indicating that line parameters are not supported for multi-file reads.
14. Set the "Max Concurrent File Reads" slider back to 1 and save.
15. Ask Roo to read multiple files again.
16. Verify: Roo should receive the <tool_error> for exceeding the maximum (which is now 1).

Screenshots

Context Management Settings:
[Screenshot of the slider]

---

Important

Enhances read_file tool to support concurrent multi-file reads with a new maxConcurrentFileReads setting, updating UI, settings, and error handling accordingly.

• Behavior:
  • Enhances read_file tool in readFileTool.ts to support multiple file reads using a JSON array for path parameter.
  • Introduces maxConcurrentFileReads setting in index.ts to limit concurrent file reads, defaulting to 1.
  • Disallows start_line and end_line when reading multiple files.
• UI and Settings:
  • Adds UI for maxConcurrentFileReads in ContextManagementSettings.tsx and SettingsView.tsx.
  • Updates webviewMessageHandler.ts to handle maxConcurrentFileReads setting changes.
• Error Handling:
  • Adds error messages in tools.json for invalid JSON, too many files, and line parameter misuse.
• Testing:
  • Updates tests in read-file-tool.test.ts and read-file-xml.test.ts to cover new multi-file functionality and error cases.

^{This description was created by for b1ac780. You can customize this summary. It will automatically update as commits are pushed.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: 279a6e2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[KJ7LNW]

I think this PR duplicates earlier work in #2886

[shariqriazz]

I think this PR duplicates earlier work in #2886

Yes and no lol. i had a PR before as well which implemented it via experimental mode and as per the request of mrubens i combined it with reed_file and some gui settings option, meanwhile 2886 was also running side by side and it all became confusing.
I and sam have even talked about merging the PRs ( getting the best out of both ) but we'll let you guys decide what to do now.

[KJ7LNW]

I and sam have even talked about merging the PRs ( getting the best out of both ) but we'll let you guys decide what to do now.

I am not in charge I just wanted to point it out. If you have already been working with Sam then whatever you think is best.

See the comments in #2886 for multi-file multi-read w/ XML-based tool syntax to conform with the same style as all other tools and facilitate future programmatic standardization of tool use (currently XML, but standardization would let us experiment with other formats such as JSON to see what works best) for when #2467 lands.

[daniel-lxs]

Hey @shariqriazz,
Thank you for your contribution! It seems that #2886 is implementing this feature and seems to be further along the pipeline.
Please feel free to reopen this PR if you think we should discuss this further.