fix: improve code indexing performance on low-end hardware (#8875)

[roomote]

Summary

This PR attempts to address Issue #8875 by making code indexing concurrency parameters configurable and adding a low resource mode for better performance on modest hardware.

Problem

Users with low-end hardware (4-core CPUs, 8GB RAM) experience extremely slow and unstable code indexing, with the UI becoming unresponsive and indexing often failing to complete.

Solution

This implementation provides configurable concurrency parameters and a low resource mode that significantly reduces resource usage:

New Configuration Settings

• roo-cline.codeIndex.parsingConcurrency: Maximum files parsed concurrently (default: 10, low resource: 2)
• roo-cline.codeIndex.batchProcessingConcurrency: Maximum embedding batches processed concurrently (default: 10, low resource: 2)
• roo-cline.codeIndex.maxPendingBatches: Maximum pending batches before throttling (default: 20, low resource: 5)
• roo-cline.codeIndex.fileProcessingConcurrency: Maximum files processed in file watcher (default: 10, low resource: 2)
• roo-cline.codeIndex.lowResourceMode: Toggle to enable low resource mode

Low Resource Mode

When enabled, automatically applies reduced defaults:

• Parsing concurrency: 10 → 2
• Batch processing: 10 → 2
• Pending batches: 20 → 5
• Batch size: 60 → 20
• File processing: 10 → 2

Implementation Details

• All concurrency parameters are now configurable through VSCode settings
• Settings apply to DirectoryScanner, FileWatcher, and CodeIndexServiceFactory
• Proper fallback handling for test environments
• Constants separated into normal and low resource defaults

Testing

• All existing tests pass ✅
• Manual configuration changes can be verified through VSCode settings UI

Notes for Users

• Enable Low Resource Mode in settings if you have modest hardware
• Further tune individual parameters if needed
• Monitor system resources and adjust settings accordingly

Future Improvements

• Auto-detection of system resources (not implemented in this PR)
• Enhanced progress reporting
• Better error messages for resource exhaustion

Feedback and guidance are welcome!

Closes #8875

---

Important

Adds configurable concurrency settings and a low resource mode to improve code indexing performance on low-end hardware.

• Behavior:
  • Adds configurable concurrency settings in package.json for code indexing, including parsingConcurrency, batchProcessingConcurrency, maxPendingBatches, and fileProcessingConcurrency.
  • Introduces lowResourceMode setting to automatically apply reduced defaults for low-end hardware.
• Implementation:
  • Updates constants/index.ts to define normal and low resource mode defaults for concurrency settings.
  • Modifies file-watcher.ts and scanner.ts to apply configurable settings and handle low resource mode.
  • Adjusts service-factory.ts to create instances with new settings, ensuring fallback handling in test environments.
• Misc:
  • Updates package.nls.json with descriptions for new settings.

^{This description was created by for 8b72f5a. You can customize this summary. It will automatically update as commits are pushed.}

[roomote]

Review Complete

✅ No issues found

The implementation successfully addresses issue #8875 by:

• Adding configurable concurrency parameters for code indexing
• Implementing a low resource mode with reduced defaults
• Allowing users to fine-tune individual settings
• Properly handling test environments with fallbacks
• Maintaining consistency across all affected components

The code quality is excellent and the changes are well-structured.

Follow Along on Roo Code Cloud

[roomote]

No issues found.