HTML Entity escaping/unesaping breaking diff search/replace #5633
Description
App Version
Version: 3.23.6 (39ab006)
API Provider
OpenRouter
Model Used
gpt4.1-mini
Roo Code Task Links (Optional)
No response
🔁 Steps to Reproduce
This is in reference to #5608 , #4077 and potentially others.
Based on my own experiences and reading through the mentioned ticket/PR:
There is some form of html escaping occurring between the LLM and Roo Code, I think I understand what is happening, but am not certain. The PR (5608) mentions an unescaping tool that was attempted to be merged in, but produced some potential breaking changes. Here is my understanding:
- Roo provides raw code to the LLM, which can understand, process, and return it, including html escaped entities
- The LLM will return escaped entities to the user via the tools at its disposal
- The html entities appear to be being unescaped in the process, leading to buggy and unexpected behavior.
Why this matters:
- The LLM is unable to write escaped entities. For example I had requested Roo fix linting errors, eg:
Please resolve these lint issues:
(direct paste or tell the llm to iterate through lint errors and resolve them)
28:210 Error: `'` can be escaped with `'`, `‘`, `'`, `’`. react/no-unescaped-entities
I fought the LLM for a good 10-15 minutes, with it insisting that the characters had been escaped and were escaped properly. I debugged smaller and smaller cases until I realized that it was incapable of replacing content with escaped entities unless it put them inside backticks.
I have used tasks like this in the past to "resolve all lint errors" and allowed the LLM to iterate until complete. I don't know when this was changed (and assume there is a good reason why, as noted in 5608), but this makes previous use cases impossible in the current state.
- If the LLM searches for escaped entities, those are also unescaped during tool use which leads to failed search cases. For example:
- I had a file with an escaped character, let's say it's "Hello world, Roo is an AI's best friend!"
- I instruct the LLM to make 'Roo" bold by adding tags around it
- The search/apply diff will:
A. Fail due to < 100% match
B. If tolerance is lower, the AI guesses and messes around, the edit may succeed, but in doing so will unescape the html entity (eg "Hello world, Roo is an AI's best friend")
In either case, the behavior is unexpected and unreliable. For A, things just fail. I realize that this is very likely what I have run into occasionally for the last month. I thought it was just LLMs being incompetent, hallucinating etc. Tool call failure after tool call failure. For B, we are talking about LLMs literally modifying unapproved sections of code/content by accident, potentially leading to build errors.
Needless to say, from a pragmatic, perspective, this is a huge bug, and while fixing it may have other side effects to consider (which I will admit, I don't understand what they are), I have had several conversations with people who have experienced seeming "unreliability" of LLMs and had believed it to be either models being dumbed down, or the Roo system prompt simply growing too large. This is something that I imagine has cost many people a lot of time and a nonzero dollar amount worth of wasted tokens due to repeated, continual tool call failures.
Thank you in advance for the thoughtful consideration and the work everyone puts into Roo!
💥 Outcome Summary
LLMs return HTML escaped entities, they are unescaped, leading to failed searches or unintended modified content.