ci: restore rule review as merge gate with defense-in-depth quality enforcement#3725
ci: restore rule review as merge gate with defense-in-depth quality enforcement#3725
Conversation
…nforcement Addresses quality concerns raised by lead dev: insufficient edge-case testing, missing regression tests on bug fixes, and review findings being silently ignored after PR #3682 made the review advisory-only. Defense-in-depth layers: 1. Agent rules: testing.md now requires edge-case tests and regression tests for all fix: PRs, with specific guidance on boundary conditions 2. CI grep enforcement: rule_lint job now rejects fix: PRs that don't add at least one new #[test] function (test-exempt label to bypass) 3. AI review gate: claude-pr-review.yml now sets a commit status (rule-review/warnings) that blocks merge when WARNINGs are found. Review prompt expanded to flag missing edge-case tests, missing regression tests, and stale documentation. review-override label provides escape hatch for legitimate exceptions. 4. GitHub ruleset updated: rule-review/warnings added as required check Key difference from pre-#3682: no per-finding /ack ceremony, no re-acking on force-push. Instead: fix warnings OR maintainer adds review-override label. Lighter friction, same enforcement. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rule Review: Minor stale docs and a logic edge caseRules checked: WarningsNone. Info
Advisory review against |
Problem
PR #3682 made the AI rule review fully advisory — findings are posted as a comment but nothing blocks merge. This led to PR #3703 merging with 4 unaddressed WARNING findings (magic numbers, stale docs, tautological assert). More broadly, there's insufficient edge-case test coverage and missing regression tests on bug fixes.
Approach
Defense-in-depth — four layers so no single failure lets bad code through:
Layer 1: Agent rules (prevent bad code from being written)
testing.mdnow requires edge-case tests and regression tests for allfix:PRsAGENTS.mdnow has a "WHEN fixing a bug" section requiring reproduce-first testingLayer 2: CI grep enforcement (catch missing tests mechanically)
rule_lintjob now rejectsfix:PRs that don't add at least one new#[test]functiontest-exemptlabel available for legitimate exceptions (must include justification comment)Layer 3: AI review gate (catch nuanced issues)
claude-pr-review.ymlnow sets a commit status (rule-review/warnings) based on WARNING findingsreview-overridelabel from maintainerfix:PRs without regression tests.claude/rules/orAGENTS.mdLayer 4: GitHub ruleset (enforce the gate)
rule-review/warningsadded as required status checkKey difference from pre-#3682
No per-finding
/ackceremony, no re-acking on force-push. Instead: fix warnings OR maintainer addsreview-overridelabel. Lighter friction, same enforcement.Testing
fix:PR without testsreview-overridelabel → workflow re-runs → sets status to success[AI-assisted - Claude]