-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Problem
"AUTH-001 covers 23/25 code paths" is meaningless without a formal definition of what a "code path" is and what "covered" means. The LEARNING-INJECTION-SPEC.md and the current simulation both gesture at coverage without defining it. Without a formal definition, coverage numbers are vibes. This ticket specifies coverage in terms that produce deterministic, comparable metrics.
Core Definitions
Scope Declaration
Every WASM module declares its scope in the manifest. The scope defines what the invariant is responsible for checking:
# In auth-001.manifest.yml
scope:
file_patterns: ["src/auth/**", "src/api/**", "pages/api/**"]
languages: [typescript, javascript]
ast_node_types: [CallExpression, AssignmentExpression] # optional narrowing
exclude: ["**/*.test.ts", "**/*.spec.ts", "**/*.mock.ts"]Coverage Unit
A coverage unit is: one file matching the scope declaration at a point in time (repo@commit).
- If
src/auth/login.tsmatchessrc/auth/**at commitabc123, that is one coverage unit. - If the file doesn't exist yet, it is not a coverage unit (not in scope = not a gap).
Coverage States
Each (invariant, file, commit) triple has one of:
| State | Meaning |
|---|---|
verified_pass |
WASM ran, verdict = pass |
verified_fail |
WASM ran, verdict = fail |
not_applicable |
WASM ran, verdict = not_applicable (file in scope but no relevant patterns) |
pending |
File matches scope, WASM not yet run against this commit |
stale |
WASM ran at older commit; file has changed since |
Coverage Score
coverage_score = (verified_pass + verified_fail + not_applicable) / (all_scope_matched_files)
A verified_fail still counts as covered — coverage measures "has the invariant run here?" not "does this pass?".
Coverage Dashboard Entry
{
"invariant_id": "AUTH-001",
"invariant_version": "1.2.0",
"as_of_commit": "def456",
"coverage_score": 0.92,
"breakdown": {
"verified_pass": 11,
"verified_fail": 1,
"not_applicable": 5,
"pending": 1,
"stale": 2
},
"uncovered": [
{ "file": "src/api/webhooks.ts", "state": "pending", "reason": "added in this commit, not yet run" }
],
"active_in_stories": ["#45", "#67", "#89"],
"stories_pending_verification": ["#89"]
}Story → Coverage Linkage
A story is linked to an invariant when:
- The story body references the invariant ID (
AUTH-001), OR - A commit in the story's PR touches a file that matches the invariant's scope, AND the invariant has been run against that commit
Issue #89 (J-AUTH-REFRESH)
└── touches: src/auth/refresh.ts (in AUTH-001 scope)
└── AUTH-001 state: pending (not yet run against this PR's commits)
└── ⚠️ story is in scope for AUTH-001 but verification pending
This is the "uncovered story" warning that board-auditor surfaces.
Invariants
| ID | Rule |
|---|---|
| COV-001 | A file added to a scope-matched path MUST appear as pending within one ruvector sync |
| COV-002 | Coverage score MUST be recomputed on every new WASM run, not cached |
| COV-003 | not_applicable MUST count toward coverage (it means the invariant consciously assessed and cleared the file) |
| COV-004 | A story touching a scope-matched file with no WASM run MUST surface as stories_pending_verification |
Acceptance Criteria
- Scope declaration YAML format defined and schema-validated
- Coverage unit, states, and score formula documented in
docs/ruvector/coverage-model.md - Coverage dashboard JSON structure defined with golden fixture
- Story → coverage linkage logic specced (how ruvector determines which stories are in scope)
- COV-001 through COV-004 invariants with enforcement strategy noted
Depends on
RUV-001 (parent)
RUV-002 (context bundle — WASM returns coverage as part of InvariantResult)