Skip to content

Commit c7f391f

Browse files
marcusgollclaude
andcommitted
feat: add automatic regression test generation on bug discovery
- Add regression-test-generator skill with framework templates (Jest, Vitest, pytest, Playwright) - Add CLI script for automation (.spec-flow/scripts/bash/regression-test-generator.sh) - Update /debug command with Step 3.5: auto-generate regression tests after root cause analysis - Update /implement command to auto-invoke /debug on test failures - Update continuous-checks.sh with Check 7/7: auto-debug trigger on test failures - Update error-log-template.md with regression test reference field Regression tests are now automatically generated when: 1. /debug is run manually (Step 3.5 generates test after root cause identified) 2. Tests fail during /implement (auto-invokes /debug) 3. Tests fail during continuous checks (triggers /debug) Tests follow Arrange-Act-Assert pattern, link to error-log.md, and prevent bug recurrence. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 5d7917b commit c7f391f

File tree

7 files changed

+1618
-3
lines changed

7 files changed

+1618
-3
lines changed

.claude/commands/phases/debug.md

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,72 @@ python .spec-flow/scripts/spec-cli.py debug "$ARGUMENTS"
134134
- `test.log` — Frontend test failures
135135
- `deploy-diag.txt` — Deployment diagnostics (if --deploy-diag used)
136136

137+
### Step 3.5: Generate Regression Test (Auto)
138+
139+
After identifying root cause, automatically generate a regression test to prevent bug recurrence.
140+
141+
**Extract bug details**:
142+
- Error ID (from error-log.md entry number, e.g., `ERR-XXXX`)
143+
- Title (brief description of the bug)
144+
- Symptoms (observable behavior from logs)
145+
- Root cause (from 5 Whys analysis in Step 3)
146+
- Component (affected file:function from stack trace)
147+
148+
**Generate regression test**:
149+
150+
```bash
151+
.spec-flow/scripts/bash/regression-test-generator.sh \
152+
--error-id "ERR-XXXX" \
153+
--title "Brief bug description" \
154+
--symptoms "Observable error behavior" \
155+
--root-cause "Why the bug occurred" \
156+
--component "src/path/file.ts:functionName" \
157+
--feature-dir "$FEATURE_DIR"
158+
```
159+
160+
**Present to user for review**:
161+
162+
```
163+
=== Regression Test Generated ===
164+
165+
Error: ERR-XXXX - {title}
166+
File: tests/regression/regression-ERR-XXXX-slug.test.ts
167+
168+
[Generated test code displayed]
169+
170+
This test will:
171+
- Capture the bug scenario to prevent regression
172+
- Fail before fix (proves bug exists)
173+
- Pass after fix (validates solution)
174+
175+
What would you like to do?
176+
[A] Save test and continue
177+
[B] Edit test before saving
178+
[C] Skip generation (add to tech debt)
179+
```
180+
181+
**User response handling**:
182+
- **A (Save)**: Test file already written by script, update error-log.md with reference
183+
- **B (Edit)**: Allow user to modify, then save updated version
184+
- **C (Skip)**: Log skip reason in NOTES.md as technical debt
185+
186+
**Update error-log.md** with regression test reference:
187+
188+
```markdown
189+
**Regression Test**:
190+
- **File**: `tests/regression/regression-ERR-XXXX-slug.test.ts`
191+
- **Status**: Generated
192+
- **Validates**: {what the test checks}
193+
```
194+
195+
**Verify test behavior**:
196+
- If bug not yet fixed: Test should FAIL (proves bug exists)
197+
- If bug already fixed: Test should PASS (validates fix)
198+
199+
**Skip conditions**:
200+
- Skip if `--from-optimize` with `--non-interactive` (auto-fix mode)
201+
- Skip if user explicitly declines with option C
202+
137203
### Step 4: Present Results to User
138204

139205
**Summary format**:
@@ -224,6 +290,12 @@ Issue {issue_id}: {resolved|requires manual fix}
224290
6. **Structured mode completion** (if --from-optimize):
225291
- Control returned to /optimize with resolution status
226292
- Issue metadata preserved in debug-session.json
293+
294+
7. **Regression test generated** (unless skipped):
295+
- Test file created in `tests/regression/` or project test location
296+
- Test follows Arrange-Act-Assert pattern
297+
- Test linked in error-log.md entry
298+
- Test staged for commit (or skip reason logged in NOTES.md)
227299
</success_criteria>
228300

229301
<verification>

.claude/commands/phases/implement.md

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1179,7 +1179,32 @@ Recommended next steps:
11791179
Failed tests:
11801180
{List failed tests}
11811181
1182-
Next: /debug to investigate and fix failures
1182+
Auto-invoking /debug to investigate failures and generate regression tests...
1183+
```
1184+
1185+
**Auto-invoke /debug** with test failure context:
1186+
1187+
```bash
1188+
# Extract failed test info and invoke /debug
1189+
/debug $FEATURE_SLUG --type=test --component={backend|frontend} --non-interactive
1190+
```
1191+
1192+
This will:
1193+
1. Run verification checks on the failing component
1194+
2. Analyze test failure logs
1195+
3. **Auto-generate regression test** for the bug (Step 3.5)
1196+
4. Update error-log.md with failure details
1197+
5. Return with actionable fix recommendations
1198+
1199+
**After /debug completes**, present results:
1200+
1201+
```
1202+
Debug session complete for test failures
1203+
1204+
Error log updated: specs/{slug}/error-log.md (Entry #N)
1205+
Regression test generated: tests/regression/regression-ERR-XXXX-{slug}.test.ts
1206+
1207+
Fix the identified issues, then re-run /implement to continue.
11831208
```
11841209

11851210
**If tasks blocked:**

0 commit comments

Comments
 (0)