Add adaptive worker fan-out and validation caps (#227)

* Improve worker validation and project fan-out caps

* fix(worker): harden validation-evidence gate + stop it hard-failing tasks

The pre-review worker validation gate (INT-2446 fan-out follow-up) had several
false-positive/false-negative edges and could kill an approvable task:

- commandLooksLikeValidation short-circuited on a leading inspection verb, so a
  chained command like `git diff && npm test` was judged as "no validation".
  Now each shell segment is evaluated independently.
- .mts/.cts modules slipped both the gate and the tester-skip check; added them
  to the relevant/code file patterns.
- Pure data trees (locale/fixtures/snapshots/mocks) forced a build/test. They
  are now exempt — but ONLY for non-code assets, so a real source module under
  those dirs (src/__mocks__/api.ts) still gets gated.
- README/LICENSE/CHANGELOG/NOTICE were treated as docs regardless of extension,
  so a source module named readme.ts skipped the gate while isTesterCodeFile
  still saw it as code. The doc-only match now requires a doc extension.
- In a tester-less pipeline a worker that reported commands=[] (e.g. git-detected
  changes promoted to success with no JSON block) was bounced twice then
  hard-failed, never reaching the reviewer. The gate now nudges once, then DEFERS
  to the reviewer instead of failing — via a pure predicate, since
  shouldAbortSelfRepair marks the session failed as a side effect.

Adds workerValidationEvidence.test.ts and a defer-to-reviewer pipeline test.

* fix(fanout): actually execute fan-out on dirty worktrees + surface it

The fan-out gate recommended fan-out 53x in the live daemon but it ran 0x: the
gate scores past its threshold almost only on self-repair retry signals
(iteration > 1), yet runWorkerFanout hard-required a clean worktree and bailed
before launching. On a retry the worktree always holds the previous iteration's
uncommitted edits, so the two mechanisms were mutually exclusive — the headline
feature never ran in production.

- Drop the clean-worktree precondition. Instead snapshot the project's
  uncommitted state (tracked + untracked) into a throwaway temp index without
  mutating the real index, seed+commit it into each sandbox, and promote only
  the incremental winner diff so it layers onto the already-dirty project.
- The fan-out onLog only reached broadcastEvent, so stdout showed the gate's
  "recommend fan-out" line but no execution evidence — the exact reason this
  was invisible. Log executing/promoted/fallback to console.log too.

Adds workerFanout.test.ts covering fan-out over a dirty worktree.

Author: unohee

config.example.yaml

@@ -47,6 +47,9 @@ autonomous:
   schedule: "*/15 * * * *"  # Every 15 minutes
   maxAttempts: 3
   maxConcurrentTasks: 4  # Number of concurrent tasks
+  worktreeMode: true  # Required for same-project parallel tasks
+  allowSameProjectConcurrent: true
+  maxConcurrentPerProject: 2  # Optional cap for same-project worktree parallelism
 
   allowedProjects:
     - ~/dev/your-project

src/agents/pairPipeline.test.ts

@@ -58,7 +58,7 @@ describe('PairPipeline model selection', () => {
       success: true,
       summary: 'done',
       filesChanged: ['src/example.ts'],
-      commands: [],
+      commands: ['npm test -- src/example.test.ts'],
       output: '',
       confidencePercent: 100,
     });
@@ -276,6 +276,220 @@ describe('PairPipeline model selection', () => {
     expect(getDefaultModel).not.toHaveBeenCalled();
   });
 
+  it('retries code-changing workers before review when validation commands are missing', async () => {
+    runWorker
+      .mockResolvedValueOnce({
+        success: true,
+        summary: 'changed code without checking it',
+        filesChanged: ['src/example.ts'],
+        commands: [],
+        output: '',
+        confidencePercent: 95,
+      })
+      .mockResolvedValueOnce({
+        success: true,
+        summary: 'changed code and ran a focused test',
+        filesChanged: ['src/example.ts'],
+        commands: ['npm test -- src/example.test.ts'],
+        output: 'PASS src/example.test.ts',
+        confidencePercent: 95,
+      });
+
+    const { PairPipeline } = await import('./pairPipeline.js');
+    const pipeline = new PairPipeline({
+      stages: ['worker', 'reviewer'],
+      maxIterations: 2,
+      roles: {
+        worker: { enabled: true, model: 'worker', timeoutMs: 0 },
+        reviewer: { enabled: true, model: 'reviewer', timeoutMs: 0 },
+      },
+    });
+
+    const result = await pipeline.run(task(), process.cwd());
+
+    expect(result.success).toBe(true);
+    expect(runWorker).toHaveBeenCalledTimes(2);
+    expect(runReviewer).toHaveBeenCalledTimes(1);
+    expect(runWorker.mock.calls[1][0]).toEqual(expect.objectContaining({
+      previousFeedback: expect.stringContaining('validation evidence missing'),
+    }));
+  });
+
+  it('defers to the reviewer instead of hard-failing when validation evidence stays missing', async () => {
+    // A worker whose git changes were promoted to success with commands=[]
+    // (no JSON block) must not be killed after a couple of retries — the gate is
+    // a nudge, so once self-repair stagnates the reviewer gets the final say.
+    runWorker.mockResolvedValue({
+      success: true,
+      summary: 'changed code, never self-reported commands',
+      filesChanged: ['src/example.ts'],
+      commands: [],
+      output: '',
+      confidencePercent: 95,
+    });
+
+    const { PairPipeline } = await import('./pairPipeline.js');
+    const pipeline = new PairPipeline({
+      stages: ['worker', 'reviewer'],
+      maxIterations: 4,
+      roles: {
+        worker: { enabled: true, model: 'worker', timeoutMs: 0 },
+        reviewer: { enabled: true, model: 'reviewer', timeoutMs: 0 },
+      },
+    });
+
+    const result = await pipeline.run(task(), process.cwd());
+
+    // Reviewer approves → task succeeds despite never getting validation evidence.
+    expect(result.success).toBe(true);
+    expect(runReviewer).toHaveBeenCalledTimes(1);
+  });
+
+  it('does not treat inspection-only commands as validation evidence', async () => {
+    runWorker
+      .mockResolvedValueOnce({
+        success: true,
+        summary: 'changed code after searching',
+        filesChanged: ['src/example.ts'],
+        commands: ['rg "npm test" package.json', 'git grep "cargo test"'],
+        output: '',
+        confidencePercent: 95,
+      })
+      .mockResolvedValueOnce({
+        success: true,
+        summary: 'changed code and ran a script smoke check',
+        filesChanged: ['src/example.ts'],
+        commands: ['python scripts/smoke_example.py --dry-run'],
+        output: 'ok',
+        confidencePercent: 95,
+      });
+
+    const { PairPipeline } = await import('./pairPipeline.js');
+    const pipeline = new PairPipeline({
+      stages: ['worker', 'reviewer'],
+      maxIterations: 2,
+      roles: {
+        worker: { enabled: true, model: 'worker', timeoutMs: 0 },
+        reviewer: { enabled: true, model: 'reviewer', timeoutMs: 0 },
+      },
+    });
+
+    const result = await pipeline.run(task(), process.cwd());
+
+    expect(result.success).toBe(true);
+    expect(runWorker).toHaveBeenCalledTimes(2);
+    expect(runReviewer).toHaveBeenCalledTimes(1);
+    expect(runWorker.mock.calls[1][0]).toEqual(expect.objectContaining({
+      previousFeedback: expect.stringContaining('non-validation commands'),
+    }));
+  });
+
+  it('requires validation for build and dependency manifests without extensions', async () => {
+    runWorker
+      .mockResolvedValueOnce({
+        success: true,
+        summary: 'changed runtime manifests without checking them',
+        filesChanged: ['Dockerfile', 'Makefile', 'requirements.txt', 'go.mod', 'Cargo.lock'],
+        commands: [],
+        output: '',
+        confidencePercent: 95,
+      })
+      .mockResolvedValueOnce({
+        success: true,
+        summary: 'changed manifests and ran a smoke build',
+        filesChanged: ['Dockerfile', 'Makefile', 'requirements.txt', 'go.mod', 'Cargo.lock'],
+        commands: ['npm run ci'],
+        output: 'ok',
+        confidencePercent: 95,
+      });
+
+    const { PairPipeline } = await import('./pairPipeline.js');
+    const pipeline = new PairPipeline({
+      stages: ['worker', 'reviewer'],
+      maxIterations: 2,
+      roles: {
+        worker: { enabled: true, model: 'worker', timeoutMs: 0 },
+        reviewer: { enabled: true, model: 'reviewer', timeoutMs: 0 },
+      },
+    });
+
+    const result = await pipeline.run(task(), process.cwd());
+
+    expect(result.success).toBe(true);
+    expect(runWorker).toHaveBeenCalledTimes(2);
+    expect(runReviewer).toHaveBeenCalledTimes(1);
+    expect(runWorker.mock.calls[1][0]).toEqual(expect.objectContaining({
+      previousFeedback: expect.stringContaining('requirements.txt'),
+    }));
+  });
+
+  it('still requires validation when tester is enabled but would skip manifest-only changes', async () => {
+    runWorker
+      .mockResolvedValueOnce({
+        success: true,
+        summary: 'changed package metadata without checking it',
+        filesChanged: ['package.json'],
+        commands: [],
+        output: '',
+        confidencePercent: 95,
+      })
+      .mockResolvedValueOnce({
+        success: true,
+        summary: 'changed package metadata and ran ci',
+        filesChanged: ['package.json'],
+        commands: ['npm run ci'],
+        output: 'ok',
+        confidencePercent: 95,
+      });
+
+    const { PairPipeline } = await import('./pairPipeline.js');
+    const pipeline = new PairPipeline({
+      stages: ['worker', 'tester', 'reviewer'],
+      maxIterations: 2,
+      roles: {
+        worker: { enabled: true, model: 'worker', timeoutMs: 0 },
+        tester: { enabled: true, model: 'tester', timeoutMs: 0 },
+        reviewer: { enabled: true, model: 'reviewer', timeoutMs: 0 },
+      },
+    });
+
+    const result = await pipeline.run(task(), process.cwd());
+
+    expect(result.success).toBe(true);
+    expect(runWorker).toHaveBeenCalledTimes(2);
+    expect(runReviewer).toHaveBeenCalledTimes(1);
+    expect(runWorker.mock.calls[1][0]).toEqual(expect.objectContaining({
+      previousFeedback: expect.stringContaining('validation evidence missing'),
+    }));
+  });
+
+  it('allows docs-only workers to reach review without validation commands', async () => {
+    runWorker.mockResolvedValueOnce({
+      success: true,
+      summary: 'updated docs',
+      filesChanged: ['docs/usage.md'],
+      commands: [],
+      output: '',
+      confidencePercent: 95,
+    });
+
+    const { PairPipeline } = await import('./pairPipeline.js');
+    const pipeline = new PairPipeline({
+      stages: ['worker', 'reviewer'],
+      maxIterations: 1,
+      roles: {
+        worker: { enabled: true, model: 'worker', timeoutMs: 0 },
+        reviewer: { enabled: true, model: 'reviewer', timeoutMs: 0 },
+      },
+    });
+
+    const result = await pipeline.run(task({ title: 'Update docs' }), process.cwd());
+
+    expect(result.success).toBe(true);
+    expect(runWorker).toHaveBeenCalledTimes(1);
+    expect(runReviewer).toHaveBeenCalledTimes(1);
+  });
+
   it('degrades to an undefined model when getDefaultModel fails', async () => {
     getDefaultModel.mockRejectedValue(new Error('no auth'));
     const { PairPipeline } = await import('./pairPipeline.js');