Blown context window with long lines

[robottwo]

App Version

3.23.12

API Provider

Anthropic

Model Used

Claude 4 sonnet

Roo Code Task Links (Optional)

No response

🔁 Steps to Reproduce

There’s a bug in Roo Code when it collects terminal output to put in the context window. Presumably, this problem is also present in the file reader tool, although I didn’t test that.

Roughly: if a terminal command outputs one long line with no new lines, Roo will include that in the context, even if it is far beyond the context window size.

This can happen easier that you might think: Roo will often run ad-hoc python or shell scripts on the terminal as part of its debugging process. I saw it stuck where it kept trying to run a command which output 16 million tokens on a single line, which exploded the context window to 16 million tokens.

This can be bad, because not only does it not run, and can’t compact, but it actually crashes Roo, and Roo will then crash when you restart it and try to load that task to continue it … so you basically lose all the history in the task for practical purposes.

💥 Outcome Summary

Expected context from shell command to get capped at a certain token size, but instead it loaded effectively unlimited tokens as long as there were no newlines.

📄 Relevant Logs or Errors (Optional)

[roomote]

I am working on fixing this issue. The problem is that terminal output collection does not properly limit context size for long lines without newlines, which can cause context window explosion and application crashes. I will investigate the terminal output handling code and implement proper token/character limits.

[roomote]

Fixed in PR #5777

Added character-based limits to terminal output truncation to prevent context window explosion. Character limits take priority over line limits and are configurable via terminalOutputCharacterLimit setting (default: 100,000 characters).

#5777