Skip to content

fix: auto mode gets stuck in validation loop on pre-existing test failures#245

Merged
rubenmarcus merged 2 commits intomainfrom
fix/auto-mode-validation-loop
Feb 23, 2026
Merged

fix: auto mode gets stuck in validation loop on pre-existing test failures#245
rubenmarcus merged 2 commits intomainfrom
fix/auto-mode-validation-loop

Conversation

@rubenmarcus
Copy link
Copy Markdown
Member

Summary

Three targeted fixes for the auto mode validation loop issue:

  1. Skip test commands in auto mode — only run build/lint validation. Pre-existing test failures (e.g., missing jsdom dep) would cause infinite loops where the agent keeps trying to fix unrelated tests. Tests remain available via --validate in non-auto mode.

  2. Clean .ralph/ between batch tasks — prevents stale iteration logs and activity data from previous tasks contaminating the next task's context.

  3. Set skipPlanInstructions in auto mode — resolves conflicting instructions where context-builder says "study IMPLEMENTATION_PLAN.md" but task-executor says "do NOT implement items from IMPLEMENTATION_PLAN.md".

Root Cause

When auto mode runs npm test and tests fail due to pre-existing structural issues (missing dependencies, wrong test environments), the agent enters an infinite loop trying to fix unfixable tests instead of moving on after completing the implementation.

Test plan

  • Run ralph-starter auto on a project with pre-existing test failures — should complete tasks without looping
  • Verify build/lint validation still runs in auto mode
  • Verify .ralph/ is clean at the start of each task
  • Verify no conflicting IMPLEMENTATION_PLAN.md instructions in auto mode

Closes #209

🤖 Generated with Claude Code

…lures (#209)

Three targeted fixes for the validation loop issue:

1. Skip test commands in auto mode validation — only run build/lint to
   avoid infinite loops on pre-existing test failures the agent can't fix.
   Tests remain opt-in via --validate in non-auto mode.

2. Clean .ralph/ directory between batch tasks — prevents stale iteration
   logs and activity data from previous tasks bleeding into the next one.

3. Set skipPlanInstructions in auto mode — avoids conflicting instructions
   where context-builder says "study IMPLEMENTATION_PLAN.md" but
   task-executor says "do NOT implement items from IMPLEMENTATION_PLAN.md".

Closes #209

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions bot added bug Something isn't working candidate-release PR is ready for release core labels Feb 23, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 23, 2026

✔️ Bundle Size Analysis

Metric Value
Base 1848.54 KB
PR 1850.44 KB
Diff 1.90 KB (0%)
Bundle breakdown
156K	dist/auth
28K	dist/automation
4.0K	dist/cli.d.ts
4.0K	dist/cli.d.ts.map
16K	dist/cli.js
12K	dist/cli.js.map
460K	dist/commands
28K	dist/config
4.0K	dist/index.d.ts
4.0K	dist/index.d.ts.map
4.0K	dist/index.js
4.0K	dist/index.js.map
536K	dist/integrations
84K	dist/llm
732K	dist/loop
172K	dist/mcp
32K	dist/presets
92K	dist/setup
36K	dist/skills
392K	dist/sources
76K	dist/ui
84K	dist/utils
336K	dist/wizard

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Feb 23, 2026

Greptile Summary

This PR addresses the auto mode validation loop issue with three targeted fixes. The changes skip test commands during auto mode validation (keeping only build/lint checks), clean the .ralph/ directory between batch tasks to prevent stale context contamination, and set skipPlanInstructions: true to resolve conflicting instructions about IMPLEMENTATION_PLAN.md.

Key Changes:

  • executor.ts:492-496 - Filter out test commands from validation in auto mode to prevent infinite loops when pre-existing test failures (like missing dependencies) are unfixable
  • task-executor.ts:106-110 - Clean .ralph/ directory between tasks to avoid stale iteration logs and activity data
  • task-executor.ts:146 - Set skipPlanInstructions: true to eliminate conflicting instructions where context-builder says "study IMPLEMENTATION_PLAN.md" but task-executor says "do NOT implement items from IMPLEMENTATION_PLAN.md"

The implementation is clean, focused, and directly addresses the root cause without over-engineering.

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The changes are well-scoped surgical fixes targeting a specific infinite loop bug. Each change has clear rationale: filtering test commands prevents unfixable test failures from looping, cleaning .ralph/ prevents context pollution, and skipPlanInstructions eliminates instruction conflicts. The modifications are minimal, non-breaking, and follow existing patterns in the codebase
  • No files require special attention

Important Files Changed

Filename Overview
src/loop/executor.ts Added test command filtering in auto mode to prevent infinite loops on pre-existing test failures (lines 492-496)
src/loop/task-executor.ts Added .ralph/ directory cleanup between batch tasks and set skipPlanInstructions: true to avoid conflicting instructions

Last reviewed commit: 5707243

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5707243592

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Address reviewer feedback:
- Codex P1: only skip test validation in batch-auto mode (not fix command)
- Codex P2: clean only transient .ralph files, preserve config.yaml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 77040cbfc1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@rubenmarcus rubenmarcus merged commit c7ae302 into main Feb 23, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working candidate-release PR is ready for release claude-code-assisted core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: auto mode gets stuck in validation loop when tests have pre-existing/structural failures

1 participant