Skip to content

Commit 6c421d2

Browse files
docs: add TASK-140 for parallel test runner
Full test suite takes several minutes sequentially. Task specifies options for parallel execution: - GNU parallel - xargs -P - Makefile with dependencies Expected speedup: 3+ minutes -> ~25-50 seconds
1 parent 45eeb12 commit 6c421d2

File tree

1 file changed

+100
-0
lines changed

1 file changed

+100
-0
lines changed
Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# TASK-140: Add parallel test runner for faster CI
2+
3+
## Priority: P2 (SECONDARY)
4+
5+
## Summary
6+
7+
The full test suite in `zig/harness/` takes a long time to run sequentially.
8+
Implement a parallel test runner to speed up CI and local development feedback loops.
9+
10+
## Current State
11+
12+
- ~40 test scripts in `zig/harness/test-*.sh`
13+
- Each test is independent (uses temp DBs, no shared state)
14+
- Sequential run takes several minutes
15+
- Each individual test takes 2-10 seconds
16+
17+
## Files to Modify
18+
19+
- `zig/harness/run-all-tests.sh` (new file)
20+
- `zig/Makefile` (add `test-parallel` target)
21+
22+
## Acceptance Criteria
23+
24+
1. [ ] Create `run-all-tests.sh` that runs tests in parallel
25+
2. [ ] Use GNU parallel or xargs -P for parallelism
26+
3. [ ] Collect and summarize results at end
27+
4. [ ] Exit non-zero if any test fails
28+
5. [ ] Support `--jobs N` or `PARALLEL_JOBS` env var
29+
6. [ ] Default to `nproc` or 4 concurrent jobs
30+
7. [ ] Show progress (e.g., "12/40 tests completed")
31+
8. [ ] Preserve individual test output for debugging failures
32+
33+
## Implementation Options
34+
35+
### Option A: GNU Parallel
36+
37+
```bash
38+
#!/usr/bin/env bash
39+
find . -name 'test-*.sh' -type f | \
40+
parallel --jobs ${PARALLEL_JOBS:-$(nproc)} \
41+
--bar \
42+
--results .tmp/test-results/{/} \
43+
bash {}
44+
```
45+
46+
### Option B: xargs
47+
48+
```bash
49+
#!/usr/bin/env bash
50+
find . -name 'test-*.sh' -type f | \
51+
xargs -P ${PARALLEL_JOBS:-4} -I{} bash -c '
52+
echo "Running {}"
53+
bash {} > .tmp/test-output-$(basename {}).log 2>&1
54+
echo "Exit code: $?"
55+
'
56+
```
57+
58+
### Option C: Makefile with dependencies
59+
60+
```makefile
61+
TESTS := $(wildcard test-*.sh)
62+
test-parallel: $(TESTS:.sh=.result)
63+
%.result: %.sh
64+
@bash $< > .tmp/$@.log 2>&1 && touch $@ || (cat .tmp/$@.log; exit 1)
65+
```
66+
67+
## Considerations
68+
69+
1. **Build contention**: Multiple tests may try to rebuild Zig extension concurrently
70+
- Solution: Add `zig build` as prerequisite before parallel tests
71+
72+
2. **Temp file collisions**: Tests create temp DBs in `.tmp/`
73+
- Current: Uses `mktemp` so no collision
74+
- Verify each test uses unique temp files
75+
76+
3. **Output interleaving**: Parallel output can be confusing
77+
- Solution: Capture per-test output, show only on failure
78+
79+
4. **Resource limits**: Too many concurrent SQLite processes
80+
- Solution: Default to 4, allow override
81+
82+
## Expected Speedup
83+
84+
With 40 tests at ~5 seconds each:
85+
- Sequential: ~200 seconds (3+ minutes)
86+
- Parallel (4 jobs): ~50 seconds
87+
- Parallel (8 jobs): ~25 seconds
88+
89+
## Parent Docs / Cross-links
90+
91+
- Gap Analysis: `research/zig-cr/97-test-gap-analysis.md`
92+
- Current runner: `zig/harness/test-parity.sh` (sequential)
93+
94+
## Progress Log
95+
96+
- 2024-12-20: Created task card
97+
98+
## Completion Notes
99+
100+
(To be filled upon completion)

0 commit comments

Comments
 (0)