Skip to content

Commit b9297ef

Browse files
docs(patterns): Add STOP Gate anti-pattern from Issue #1755
Add Multi-Model Validation Anti-Pattern documenting that STOP gates help Opus but break Sonnet, and that flow language works for both. Pattern shows: - Same prompt, opposite model responses - STOP gates: Opus proceeds, Sonnet asks permission - V2 (flow language): Both models improve - 12-21% performance improvement from removing gates Empirical evidence from 6/8 benchmarks (75% complete): - Sonnet: 8/22 → 22/22 steps, -16% cost - Opus: Maintains 20/22, -21% cost Lesson: Always test multi-model before deploying AI guidance. Related: #1755, DISCOVERIES.md entry 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 2289aea commit b9297ef

File tree

1 file changed

+93
-0
lines changed

1 file changed

+93
-0
lines changed

.claude/context/PATTERNS.md

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -628,6 +628,99 @@ If ratio < 3.0, seek simpler alternatives.
628628

629629
> **Origin**: Discovered evaluating PBZFT vs N-Version Programming. PBZFT would be 6-9x more complex with zero benefit. See DISCOVERIES.md (2025-10-20).
630630
631+
## Multi-Model AI Patterns
632+
633+
### Pattern: Multi-Model Validation Anti-Pattern (STOP Gates)
634+
635+
**Challenge**: Validation checkpoints in AI guidance can trigger model-specific responses, helping one model while breaking another.
636+
637+
**Problem**: STOP gates added to improve Opus caused Sonnet degradation:
638+
639+
- Opus 4.5: STOP gates help (20/22 → 22/22 steps) ✅
640+
- Sonnet 4.5: STOP gates break (22/22 → 8/22 steps) ❌
641+
- Same text, opposite outcomes
642+
643+
**Solution**: Remove validation checkpoints, use flow language instead.
644+
645+
**Example - Bad (STOP Gates)**:
646+
647+
```markdown
648+
## Step 1: Create GitHub Issue
649+
650+
Create an issue for your feature.
651+
652+
## STOP - Verify Issue Created
653+
654+
Before proceeding to Step 2, confirm:
655+
656+
- [ ] GitHub issue created
657+
- [ ] Issue number recorded
658+
659+
Only proceed after verification complete.
660+
661+
## Step 2: Create Branch
662+
663+
...
664+
```
665+
666+
**Example - Good (Flow Language)**:
667+
668+
```markdown
669+
## Step 1: Create GitHub Issue
670+
671+
Create an issue for your feature.
672+
673+
## Step 2: Create Branch
674+
675+
After creating the issue, create a feature branch...
676+
```
677+
678+
**Why This Works**:
679+
680+
- Provides clear structure without interruption points
681+
- Uses flow language ("After X, do Y") not interruption language ("STOP before Y")
682+
- Allows continuous autonomous execution
683+
- Works for both models
684+
685+
**Empirical Evidence** (Issue #1755, 6/8 benchmarks complete):
686+
687+
| Model | With STOP Gates | Without STOP Gates (V2) |
688+
| ------ | ---------------- | --------------------------------- |
689+
| Sonnet | 8/22 steps (36%) | 22/22 steps (100%) |
690+
| Opus | 22/22 steps | ~20/22 steps (maintains baseline) |
691+
692+
**Performance Results**:
693+
694+
- Sonnet V2: -16% cost improvement
695+
- Opus V2: -21% cost improvement
696+
- Removing gates IMPROVES performance (STOP Gate Paradox)
697+
698+
**Key Points**:
699+
700+
- Different models interpret "STOP" differently
701+
- Opus: Treats as checkpoint, proceeds
702+
- Sonnet: Treats as permission gate, asks user
703+
- High-salience language ("STOP", "MUST", ALL CAPS) risky
704+
- Always test multi-model before deploying guidance changes
705+
706+
**When to Use Flow Language**:
707+
708+
- "After X, proceed to Y" ✅
709+
- "When X completes, Y begins" ✅
710+
- "Following X, continue with Y" ✅
711+
712+
**When to AVOID Interruption Language**:
713+
714+
- "STOP before Y" ❌
715+
- "Only proceed after X" ❌
716+
- "Wait for confirmation before Y" ❌
717+
718+
**Related**: Issue #1755, DISCOVERIES.md (2025-12-01)
719+
**Validation**: 75% complete (6/8 benchmarks), both models tested
720+
**Impact**: $20K-$406K annual savings from removing STOP gates
721+
722+
---
723+
631724
## Remember
632725

633726
These patterns represent proven solutions from real development challenges:

0 commit comments

Comments
 (0)