Diff tool incorrectly treats < and &lt; as identical

[WAcry]

App Version

3.28.1

API Provider

OpenAI Compatible

Model Used

GPT 5

Roo Code Task Links (Optional)

No response

🔁 Steps to Reproduce

Description:
When applying diffs to files, the diff tool incorrectly normalizes < and < as equivalent, causing false positives where no changes are detected. As a result, valid diffs that replace escaped generic type syntax (<...>) with real generic type syntax (<...>) are ignored.

Steps to Reproduce:

1. Have a file containing escaped generics:

   .FilterBatch<int>(batch => batch.Count == 3)
   .MapBatch<int, int>(batch => batch.Sum())

2. Apply a diff that replaces < / > with < / >:

LLM is suggesting:

<![CDATA[
<<<<<<< SEARCH
                    .FilterBatch&lt;int&gt;(batch =&gt; batch.Count == 3)
=======
                    .FilterBatch<int>(batch => batch.Count == 3)
>>>>>>> REPLACE
]]>

<![CDATA[
<<<<<<< SEARCH
                    .MapBatch&lt;int, int&gt;(batch =&gt; batch.Sum())
=======
                    .MapBatch<int, int>(batch => batch.Sum())
>>>>>>> REPLACE
]]>

3. The diff tool reports:

Diff 1: Search and replace content are identical - no changes would be made

Debug Info:
- Search and replace must be different to make changes
- Use read_file to verify the content you want to change
tests/TorqueFlow.Tests/DslTests.cs - Diff 2: Search and replace content are identical - no changes would be made

Debug Info:
- Search and replace must be different to make changes
- Use read_file to verify the content you want to change

💥 Outcome Summary

Expected Behavior:
The tool should recognize < and < (likewise > and >) as different tokens, and apply the replacement accordingly.

Actual Behavior:
The diff tool normalizes them as equivalent, leading to false negatives and preventing legitimate replacements.

📄 Relevant Logs or Errors (Optional)

[roomote]

I'm exploring a potential fix for the HTML entity normalization issue in the diff tool. The problem appears to be that < and < (as well as > and >) are being treated as identical when they should be recognized as different. I'll work on ensuring the diff tool properly distinguishes between HTML entities and their literal characters.

[roomote]

I've successfully implemented a fix for this issue and opened PR #7965: #7965

The fix ensures that HTML entities like < and > are properly distinguished from their literal characters < and > during diff comparison. The solution stores the original content before any transformations and uses it for the identity check, preserving the distinction between HTML entities and their literal characters.

All CI checks have passed ✅:

• All new tests pass (7/7)
• All existing tests pass (60/60)
• Linting and type checking pass
• Integration tests pass

The implementation has been reviewed with a 95% confidence score and is ready for merge.

[daniel-lxs]

Thank you for reporting this issue. This is actually a known issue that's already being tracked in #4077 - the diff tool's handling of HTML entities (<, >, &, etc.) during search and replace operations.

The current behavior where HTML entities are automatically unescaped during diff operations is intentional, though it does cause problems in certain scenarios like yours where you need to replace escaped generic type syntax with real generic type syntax.

As explained in the discussion on #4077 and #5633, there's a proposed solution to add a user setting that would allow control over HTML entity escaping behavior, which would be tied to the selected profile/model. This way, users could enable/disable automatic unescaping depending on their needs.

I'm closing this as a duplicate of #4077. Please follow that issue for updates on the resolution.

Related issues:

• Why does it remove 'amp;' from the SEARCH code during the apply_diff process? #4077 (main tracking issue - still open)
• HTML Entity escaping/unesaping breaking diff search/replace #5633 (closed in favor of Why does it remove 'amp;' from the SEARCH code during the apply_diff process? #4077)

[daniel-lxs]

Closing as duplicate of #4077. See #7964 (comment) for details.