Skip to content

Commit a47247d

Browse files
josecelanojosecelano
committed
docs: add coverage & reporting EPIC (#85) with child issue specifications
- Create EPIC issue #85 for coverage improvements - Add issue #86: coverage check to pre-commit script - Add issue #87: coverage CI workflow - Add issue #88: refactor testing documentation structure - Add issue #89: write coverage documentation - Update project-words.txt with browsable, Swatinem, taiki
1 parent c6bc587 commit a47247d

6 files changed

+1246
-0
lines changed

‎docs/issues/85-epic-coverage-and-reporting.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
Title: Coverage & Reporting EPIC - Test coverage checks, workflows, and docs
2+
3+
**Issue**: #85
4+
5+
## Overview
6+
7+
## Overview
8+
9+
This epic tracks improving test coverage visibility and guardrails across the project. The goal is to ensure coverage is visible, actionable, and enforced where appropriate so we keep regressions small and reviewers aware of coverage impacts.
10+
11+
This epic groups related tasks for coverage reporting infrastructure:
12+
13+
- Add coverage check to pre-commit script (non-blocking, runs last after all other checks)
14+
- Create a CI workflow to generate coverage reports and upload artifacts
15+
- Refactor testing documentation into organized folder structure
16+
- Write new coverage documentation in the refactored structure
17+
18+
**Implementation Status:**
19+
20+
- ✅ The `cov-check` alias has been added to `.cargo/config.toml` using the native `cargo llvm-cov --fail-under-lines 85` option
21+
- ⏳ The four tasks above are specified in this EPIC and will be implemented through separate issues
22+
23+
## Roadmap Reference
24+
25+
From [docs/roadmap.md](../roadmap.md):
26+
27+
> See "Development Process" guidance for creating roadmap tasks and epics. This EPIC adds quality tooling around tests and CI reporting to the roadmap's implementation practices.
28+
29+
## Tasks
30+
31+
- [ ] Add coverage check to pre-commit script (non-blocking, informational)
32+
- [ ] Create a coverage workflow to generate the coverage report
33+
- [ ] Refactor testing documentation structure into organized folder
34+
- [ ] Write documentation about coverage (expectations, how-to, PR acceptance criteria)
35+
36+
(Detailed task specifications exist as child issue specifications in `docs/issues/`.)
37+
38+
### Child Issue Specifications
39+
40+
1. [#86 - `86-add-coverage-check-to-pre-commit.md`](./86-add-coverage-check-to-pre-commit.md) - Add `cargo cov-check` to pre-commit STEPS array
41+
2. [#87 - `87-create-coverage-ci-workflow.md`](./87-create-coverage-ci-workflow.md) - Create GitHub Actions workflow for coverage reporting
42+
3. [#88 - `88-refactor-testing-documentation-structure.md`](./88-refactor-testing-documentation-structure.md) - Split testing.md into organized folder structure
43+
4. [#89 - `89-write-coverage-documentation.md`](./89-write-coverage-documentation.md) - Create coverage.md in refactored testing docs
44+
45+
**Implementation Order:**
46+
47+
Tasks 1 and 2 can be done independently. Task 3 must be completed before Task 4 (documentation refactoring before adding new coverage docs).
48+
49+
## Goals
50+
51+
- Improve visibility of test coverage across the repository
52+
- Prevent accidental, unnoticed coverage regressions
53+
- Make it easy for contributors to generate and inspect coverage locally
54+
55+
## Acceptance Criteria
56+
57+
- [ ] EPIC specification exists in `docs/issues/` and is reviewed
58+
- [ ] The four child task specifications are clear and scoped to be implemented independently
59+
- [ ] `cov-check` alias is available in `.cargo/config.toml` using native `--fail-under-lines 85`
60+
- [ ] Pre-commit script includes coverage check (non-blocking, informational only)
61+
- [ ] Testing documentation is organized in `docs/contributing/testing/` folder structure
62+
- [ ] Coverage documentation exists at `docs/contributing/testing/coverage.md`
63+
- [ ] Documentation explains how coverage is measured and how to run the local coverage command (references `.cargo/config.toml` aliases)
64+
65+
## Related
66+
67+
- Parent: #1 (Project Roadmap)
68+
- See project coverage aliases in `.cargo/config.toml` (`cov`, `cov-lcov`, `cov-html`, `cov-codecov`)
69+
70+
---
71+
72+
Notes:
73+
74+
- This EPIC includes four child issues with detailed specifications in `docs/issues/`
75+
- The `cov-check` alias was implemented immediately using cargo llvm-cov's native `--fail-under-lines` option rather than a custom script
76+
- Coverage check in pre-commit is non-blocking to allow urgent patches/fixes even if coverage temporarily drops below threshold
77+
- Testing documentation refactoring (task 3) must be completed before coverage documentation (task 4) to avoid mixing refactoring with new content

‎docs/issues/86-add-coverage-check-to-pre-commit.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# Add Coverage Check to Pre-commit Script
2+
3+
**Issue**: #86
4+
**Parent Epic**: #85 - Coverage & Reporting EPIC
5+
**Related**: [85-epic-coverage-and-reporting.md](./85-epic-coverage-and-reporting.md)
6+
7+
## Overview
8+
9+
Add code coverage validation to the pre-commit script (`scripts/pre-commit.sh`) as the last check in the steps array. The check uses the `cargo cov-check` alias which validates that coverage meets the 85% threshold and will fail the pre-commit if coverage drops below this level.
10+
11+
## Goals
12+
13+
- [ ] Make coverage validation part of the pre-commit process
14+
- [ ] Provide developers immediate feedback about coverage impact
15+
- [ ] Run coverage check last to ensure all other checks pass first (saves time)
16+
- [ ] Use existing step infrastructure for consistency
17+
18+
## 🏗️ Architecture Requirements
19+
20+
**DDD Layer**: Infrastructure (Tooling/Scripts)
21+
**Module Path**: `scripts/pre-commit.sh`
22+
**Pattern**: Shell script validation tool
23+
24+
### Module Structure Requirements
25+
26+
- [ ] Follow existing script structure and patterns in `scripts/pre-commit.sh`
27+
- [ ] Use consistent formatting and messaging style
28+
- [ ] Maintain the step-based execution model
29+
30+
### Architectural Constraints
31+
32+
- [ ] Must run as the last step in the `STEPS` array
33+
- [ ] Should use existing `cargo cov-check` alias from `.cargo/config.toml`
34+
- [ ] Must leverage existing step execution infrastructure (timing, error handling, formatting)
35+
- [ ] Must provide clear, actionable messaging
36+
37+
### Anti-Patterns to Avoid
38+
39+
- ❌ Adding special-case logic instead of using the existing step array
40+
- ❌ Running coverage before other checks (wastes time if linting fails)
41+
- ❌ Duplicating timing/formatting logic that already exists
42+
43+
## Specifications
44+
45+
### Current Pre-commit Structure
46+
47+
The `scripts/pre-commit.sh` script uses a step-based execution model with an array of checks:
48+
49+
```bash
50+
declare -a STEPS=(
51+
"Checking for unused dependencies (cargo machete)|No unused dependencies found|||cargo machete"
52+
"Running linters|All linters passed|||cargo run --bin linter all"
53+
"Running tests|All tests passed|||cargo test"
54+
"Testing cargo documentation|Documentation builds successfully|||cargo doc --no-deps --bins --examples --workspace --all-features"
55+
"Running comprehensive E2E tests|All E2E tests passed|(Filtering logs to WARNING level and above - this may take a few minutes)|RUST_LOG=warn|cargo run --bin e2e-tests-full"
56+
)
57+
```
58+
59+
Each step follows the format:
60+
`"description|success_message|optional_pre_message|optional_env_var|command"`
61+
62+
All steps fail fast on errors due to `set -euo pipefail`.
63+
64+
### Proposed Change
65+
66+
Add coverage check as the **last step** in the `STEPS` array:
67+
68+
```bash
69+
declare -a STEPS=(
70+
"Checking for unused dependencies (cargo machete)|No unused dependencies found|||cargo machete"
71+
"Running linters|All linters passed|||cargo run --bin linter all"
72+
"Running tests|All tests passed|||cargo test"
73+
"Testing cargo documentation|Documentation builds successfully|||cargo doc --no-deps --bins --examples --workspace --all-features"
74+
"Running comprehensive E2E tests|All E2E tests passed|(Filtering logs to WARNING level and above - this may take a few minutes)|RUST_LOG=warn|cargo run --bin e2e-tests-full"
75+
"Running code coverage check|Coverage meets 85% threshold|(Informational only - does not block commits)||cargo cov-check"
76+
)
77+
```
78+
79+
### Why This Approach?
80+
81+
**Advantages:**
82+
83+
- ✅ **Consistent**: Follows existing script patterns
84+
- ✅ **Simple**: Just one line added to the array
85+
- ✅ **Maintainable**: No special-case logic needed
86+
- ✅ **Clear**: Uses same messaging format as other steps
87+
- ✅ **Automatic timing**: Built-in timing display like other steps
88+
89+
**Behavior:**
90+
91+
- Runs last after all other checks pass
92+
- Still fails fast if coverage is below 85% (exits with non-zero code)
93+
- Shows clear error message from `cargo cov-check`
94+
- Uses existing error handling and timing infrastructure
95+
96+
### Alternative: Non-blocking Implementation
97+
98+
**Current decision**: Treat coverage like other mandatory checks (blocking).
99+
100+
**If non-blocking behavior is needed later**, the script could be modified to:
101+
102+
1. Add a separate section after the STEPS loop
103+
2. Use conditional execution (`if cargo cov-check; then ... else ... fi`)
104+
3. Exit with code 0 regardless of coverage result
105+
106+
However, the simpler approach is to start with blocking behavior and only add complexity if feedback shows it's needed.
107+
108+
## Implementation Plan
109+
110+
### Phase 1: Add Coverage Step (5 minutes)
111+
112+
- [ ] Open `scripts/pre-commit.sh`
113+
- [ ] Locate the `declare -a STEPS=()` array
114+
- [ ] Add new line at the end of the array (before the closing parenthesis)
115+
- [ ] The new line should be:
116+
117+
```bash
118+
"Running code coverage check|Coverage meets 85% threshold|(Informational only - does not block commits)||cargo cov-check"
119+
```
120+
121+
- [ ] Save the file
122+
123+
### Phase 2: Testing (10 minutes)
124+
125+
- [ ] Run `./scripts/pre-commit.sh` on current codebase (coverage should pass at 85.75%)
126+
- [ ] Verify coverage check runs last (after E2E tests)
127+
- [ ] Verify timing is displayed correctly (uses existing timing infrastructure)
128+
- [ ] Verify success message appears when coverage passes
129+
- [ ] Verify script shows clear output from `cargo cov-check`
130+
131+
## Acceptance Criteria
132+
133+
**Quality Checks**:
134+
135+
- [ ] Pre-commit checks pass: `./scripts/pre-commit.sh`
136+
137+
**Task-Specific Criteria**:
138+
139+
- [ ] Coverage check is added to `scripts/pre-commit.sh`
140+
**Task-Specific Criteria**:
141+
142+
- [ ] Coverage check added as last step in `STEPS` array in `scripts/pre-commit.sh`
143+
- [ ] Uses `cargo cov-check` alias (already exists in `.cargo/config.toml`)
144+
- [ ] Success message reads: "Coverage meets 85% threshold"
145+
- [ ] Pre-message reads: "(Informational only - does not block commits)"
146+
- [ ] Script exits with non-zero code if coverage is below 85%
147+
- [ ] Timing information is automatically displayed (inherited from step infrastructure)
148+
- [ ] All existing checks still work correctly
149+
150+
## Related Documentation
151+
152+
- [Pre-commit Process](../contributing/commit-process.md) - Pre-commit workflow
153+
- [Development Principles](../development-principles.md) - Quality standards
154+
- [Testing Conventions](../contributing/testing.md) - Testing guidelines
155+
156+
## Notes
157+
158+
- The coverage check is intentionally non-blocking to support urgent fixes and patches
159+
- Running coverage last ensures developers don't waste time waiting for coverage if linting fails
160+
- The `tail -5` filter keeps output concise while showing the essential coverage summary
161+
- Exit code 0 is important for CI/CD pipelines that may use this script

0 commit comments

Comments
 (0)