Skip to content

RUV-005: Spec — coverage formal definition: scope declarations and invariant coverage maps #26

@Hulupeep

Description

@Hulupeep

Problem

"AUTH-001 covers 23/25 code paths" is meaningless without a formal definition of what a "code path" is and what "covered" means. The LEARNING-INJECTION-SPEC.md and the current simulation both gesture at coverage without defining it. Without a formal definition, coverage numbers are vibes. This ticket specifies coverage in terms that produce deterministic, comparable metrics.

Core Definitions

Scope Declaration

Every WASM module declares its scope in the manifest. The scope defines what the invariant is responsible for checking:

# In auth-001.manifest.yml
scope:
  file_patterns: ["src/auth/**", "src/api/**", "pages/api/**"]
  languages: [typescript, javascript]
  ast_node_types: [CallExpression, AssignmentExpression]  # optional narrowing
  exclude: ["**/*.test.ts", "**/*.spec.ts", "**/*.mock.ts"]

Coverage Unit

A coverage unit is: one file matching the scope declaration at a point in time (repo@commit).

  • If src/auth/login.ts matches src/auth/** at commit abc123, that is one coverage unit.
  • If the file doesn't exist yet, it is not a coverage unit (not in scope = not a gap).

Coverage States

Each (invariant, file, commit) triple has one of:

State Meaning
verified_pass WASM ran, verdict = pass
verified_fail WASM ran, verdict = fail
not_applicable WASM ran, verdict = not_applicable (file in scope but no relevant patterns)
pending File matches scope, WASM not yet run against this commit
stale WASM ran at older commit; file has changed since

Coverage Score

coverage_score = (verified_pass + verified_fail + not_applicable) / (all_scope_matched_files)

A verified_fail still counts as covered — coverage measures "has the invariant run here?" not "does this pass?".

Coverage Dashboard Entry

{
  "invariant_id": "AUTH-001",
  "invariant_version": "1.2.0",
  "as_of_commit": "def456",
  "coverage_score": 0.92,
  "breakdown": {
    "verified_pass": 11,
    "verified_fail": 1,
    "not_applicable": 5,
    "pending": 1,
    "stale": 2
  },
  "uncovered": [
    { "file": "src/api/webhooks.ts", "state": "pending", "reason": "added in this commit, not yet run" }
  ],
  "active_in_stories": ["#45", "#67", "#89"],
  "stories_pending_verification": ["#89"]
}

Story → Coverage Linkage

A story is linked to an invariant when:

  1. The story body references the invariant ID (AUTH-001), OR
  2. A commit in the story's PR touches a file that matches the invariant's scope, AND the invariant has been run against that commit
Issue #89 (J-AUTH-REFRESH)
  └── touches: src/auth/refresh.ts (in AUTH-001 scope)
  └── AUTH-001 state: pending (not yet run against this PR's commits)
  └── ⚠️ story is in scope for AUTH-001 but verification pending

This is the "uncovered story" warning that board-auditor surfaces.

Invariants

ID Rule
COV-001 A file added to a scope-matched path MUST appear as pending within one ruvector sync
COV-002 Coverage score MUST be recomputed on every new WASM run, not cached
COV-003 not_applicable MUST count toward coverage (it means the invariant consciously assessed and cleared the file)
COV-004 A story touching a scope-matched file with no WASM run MUST surface as stories_pending_verification

Acceptance Criteria

  • Scope declaration YAML format defined and schema-validated
  • Coverage unit, states, and score formula documented in docs/ruvector/coverage-model.md
  • Coverage dashboard JSON structure defined with golden fixture
  • Story → coverage linkage logic specced (how ruvector determines which stories are in scope)
  • COV-001 through COV-004 invariants with enforcement strategy noted

Depends on

RUV-001 (parent)
RUV-002 (context bundle — WASM returns coverage as part of InvariantResult)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions