fix: resolve E2BIG error by passing large prompts via stdin to Claude CLI

[Fovty]

fix: resolve E2BIG error by passing large prompts via stdin to Claude CLI

• Pass messages via stdin instead of command line arguments to avoid Linux argument length limits
• Add --input-format text flag to claude CLI command
• Update execa configuration to use stdin pipe
• Fix corresponding unit tests with proper async iterator mocking
• Resolves spawn E2BIG errors when using very large conversation histories

Related GitHub Issue

Closes: #5064
Closes: #5097

Description

This PR fixes the spawn E2BIG error that occurs when users interact with Claude CLI through Roo Code with large conversation histories or extensive context. The error happens because Linux imposes a per-argument limit (~128 KiB) on command-line arguments passed to execve() system calls.

Key implementation details:

• Modified src/integrations/claude-code/run.ts: Replaced command-line argument passing with stdin streaming for large message payloads
• Enhanced test coverage in src/integrations/claude-code/__tests__/run.spec.ts: Fixed async iterator mocking to properly test stdin functionality
• Backward compatibility: Maintains all existing functionality

Technical approach:

• Removed JSON.stringify(messages) from args array to avoid argument length limits
• Added --input-format text flag to enable stdin message reading
• Configured execa with stdin: "pipe" and stream messages as JSON
• Ensured proper stdin termination with child.stdin.end()

Test Procedure

Unit Testing:

1. Run tests from the src/ directory: cd src && npm test src/integrations/claude-code/__tests__/run.spec.ts
2. Verify all 5 tests pass, including the new stdin functionality test
3. Confirm no regressions in existing Claude CLI integration behavior

Manual Testing (if accessible):

1. Use Roo Code with a very large conversation history (>100KB of messages)
2. Attempt to interact with Claude CLI
3. Verify no spawn E2BIG errors occur
4. Confirm normal Claude CLI responses are received

Environment Requirements:

• Linux environment (where E2BIG errors are most common)

Pre-Submission Checklist

• Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
• Scope: My changes are focused on the linked issue (one major feature/fix per PR).
• Self-Review: I have performed a thorough self-review of my code.
• Testing: New and/or updated tests have been added to cover my changes (if applicable).
• Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
• Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Documentation Updates

• No documentation updates are required.

This is an internal bug fix that doesn't change user-facing APIs or require documentation updates. The fix is transparent to end users.

Additional Notes

• Performance: Stdin streaming may have slightly better memory characteristics for very large prompts compared to command-line arguments
• Compatibility: The change is backward compatible and doesn't affect any existing functionality

Get in Touch

@Fovty

---

Important

Fix E2BIG error in Claude CLI by passing large prompts via stdin, updating run.ts and tests in run.spec.ts.

• Behavior:
  • Pass messages via stdin in run.ts to avoid E2BIG error with large prompts.
  • Add --input-format text flag to Claude CLI command.
  • Configure execa with stdin: "pipe".
• Testing:
  • Update run.spec.ts to mock stdin behavior and verify new implementation.
  • Ensure tests cover stdin functionality and maintain backward compatibility.
• Misc:
  • Remove JSON.stringify(messages) from command-line arguments in run.ts.
  • Ensure proper stdin termination with child.stdin.end() in run.ts.

^{This description was created by for eee0c20. You can customize this summary. It will automatically update as commits are pushed.}

[edgan]

This doesn't work for me. I tested it with the same prompt that caused me to create #5064, and I get this error. This is also the exact same error I was running into while trying to make my own PR in a very like way. It relates to part of the system prompt getting misinterpreted.

error: unknown option '-------'

The part of the system prompt that contains this string.

SEARCH\n:start_line: (required) The line number of original content where the search block starts.\n-------\n[exact content to find including whitespace]\n=======\n[new content to replace with]\n>>>>>>> REPLACE\n\n\n\n\nExample:\n\nOriginal file:\n\n1 | def calculate_total(items):\n2 | total = 0\n3 | for item in items:\n4 | total += item\n5 | return total\n\n\nSearch/Replace content:\n\n<<<<<<< SEARCH\n:start_line:1\n-------\ndef calculate_total(items):\n total = 0\n for item in items:\n total += item\n return total\n=======\ndef calculate_total(items):\n """Calculate total with 10% markup"""\n return sum(item * 1.1 for item in items)\n>>>>>>> REPLACE\n\n\n\nSearch/Replace content with multi edits:\n\n<<<<<<< SEARCH\n:start_line:1\n-------\ndef calculate_total(items):\n sum = 0\n=======\ndef calculate_sum(items):\n sum = 0\n>>>>>>> REPLACE\n\n<<<<<<< SEARCH\n:start_line:4\n-------\n

[daniel-lxs]

I've pushed a fix that should address the reported issue. The problem was that the --input-format text flag was causing the Claude CLI to misinterpret the system prompt argument.

When the system prompt contained certain characters (like ------- from the apply_diff tool documentation), the CLI was parsing them as command-line options, resulting in the error: error: unknown option '-------'.

Changes made:

1. Removed the --input-format text flag from the Claude CLI arguments
2. Updated the corresponding tests

The Claude CLI appears to handle JSON input via stdin correctly without needing to specify an input format. This should resolve the issue while maintaining the fix for the original E2BIG error.

Please test this with your original prompt that was causing the issue and let me know if it works correctly now.

For reference, these are the only required parameters when using stdin: https://docs.anthropic.com/en/docs/claude-code/sdk#streaming-json-input

[daniel-lxs]

Thank you @Fovty for your contribution!

LGTM

[edgan]

Same issue with the latest commit. I can see your --input-format text removal did happen.

Beginning of error:

Command failed with exit code 1: /home/ngrennan/bin/claude -p --system-prompt

End of error:
error: unknown option '-------'

[edgan]

I can confirm removing the system prompt makes the error go away, and it worked. So the problem to solve is how to format the system prompt properly. I am surprised the system prompt worked in the first place, but there is some under the hood issue with execa when you add the the .pipe.

I also suspect that you are kicking the can with the E2BIG issue by using echo. This is why I was trying to use temporary files to do a cat file1 file2 file3 method. Which completely bypasses the max command line length of 128kb.

Here are the biggest files in one of my projects:

100K	./app/src/main/java/me/edgan/redditslide/ui/settings/SettingsGeneralFragment.java
108K	./app/src/main/java/me/edgan/redditslide/Adapters/CommentAdapterHelper.java
116K	./app/src/main/java/me/edgan/redditslide/SubmissionViews/PopulateSubmissionViewHolder.java
140K	./app/src/main/java/me/edgan/redditslide/Activities/SubredditView.java
164K	./app/src/main/java/me/edgan/redditslide/Fragments/CommentPage.java

-               "--system-prompt",
-               systemPrompt,