feat: add file deduplication to reduce token usage

[MuriloFP]

Re-implements PR RooCodeInc#1374 functionality on current codebase structure

Overview

This PR re-implements the file deduplication feature originally proposed in PR RooCodeInc#1374. The original PR had significant merge conflicts and was based on an older codebase structure. This implementation provides the same functionality but integrated with the current architecture.

Changes

• Added deduplicateReadFileHistory() method to Task.ts that removes duplicate file reads from conversation history
• Implemented smart deduplication that:
  • Preserves the most recent read of each file
  • Handles partial file reads (line ranges) correctly
  • Preserves @mention file content
  • Only deduplicates within the current task's messages
• Made the feature configurable via contextDeduplication experiment flag (disabled by default)
• Added comprehensive test coverage for all deduplication scenarios

Key Improvements Over Original PR

1. Partial Read Support: Correctly handles files read with line ranges
2. @mention Preservation: Keeps user-provided file content intact
3. Configurable: Feature can be toggled via experiment flag
4. Better Integration: Works with the refactored Task.ts structure
5. Comprehensive Testing: Full test coverage for all scenarios

Testing

All new tests pass and the feature has been validated to work correctly with:

• Basic deduplication of repeated full file reads
• Handling of partial file reads with line ranges
• Preservation of @mention file content
• Multiple file deduplication
• Edge cases (empty history, no duplicates)

Closes RooCodeInc#1374

[KJ7LNW]

see also: RooCodeInc#6053 prevents duplicate reads all together.