Skip to content

Commit 8c5d940

Browse files
feat(zig): add parallel test runner for 7x faster CI
- Create run-all-tests.sh with xargs-based parallel execution - Add --fast (default), --all, --slow modes to control test selection - Exclude slow tests (fuzz, stress, large-data) by default - 39 fast tests complete in ~19s vs ~200s+ sequential - Add Makefile targets: test-parity-parallel, test-parity-all, test-slow Closes TASK-140
1 parent c6a4526 commit 8c5d940

File tree

3 files changed

+406
-14
lines changed

3 files changed

+406
-14
lines changed

.tasks/backlog/TASK-140-parallel-test-runner.md

Lines changed: 33 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -21,14 +21,14 @@ Implement a parallel test runner to speed up CI and local development feedback l
2121

2222
## Acceptance Criteria
2323

24-
1. [ ] Create `run-all-tests.sh` that runs tests in parallel
25-
2. [ ] Use GNU parallel or xargs -P for parallelism
26-
3. [ ] Collect and summarize results at end
27-
4. [ ] Exit non-zero if any test fails
28-
5. [ ] Support `--jobs N` or `PARALLEL_JOBS` env var
29-
6. [ ] Default to `nproc` or 4 concurrent jobs
30-
7. [ ] Show progress (e.g., "12/40 tests completed")
31-
8. [ ] Preserve individual test output for debugging failures
24+
1. [x] Create `run-all-tests.sh` that runs tests in parallel
25+
2. [x] Use GNU parallel or xargs -P for parallelism
26+
3. [x] Collect and summarize results at end
27+
4. [x] Exit non-zero if any test fails
28+
5. [x] Support `--jobs N` or `PARALLEL_JOBS` env var
29+
6. [x] Default to `nproc` or 4 concurrent jobs
30+
7. [x] Show progress (e.g., "12/40 tests completed")
31+
8. [x] Preserve individual test output for debugging failures
3232

3333
## Implementation Options
3434

@@ -94,7 +94,31 @@ With 40 tests at ~5 seconds each:
9494
## Progress Log
9595

9696
- 2024-12-20: Created task card
97+
- 2024-12-20: Implemented parallel test runner
9798

9899
## Completion Notes
99100

100-
(To be filled upon completion)
101+
Completed on 2024-12-20.
102+
103+
### Implementation
104+
105+
Created `zig/harness/run-all-tests.sh` with:
106+
- Parallel execution using xargs -P (portable, no GNU parallel dependency)
107+
- `--fast` mode (default): excludes slow tests (fuzz, stress, large-data)
108+
- `--all` mode: runs all tests including slow ones
109+
- `--slow` mode: runs only slow tests
110+
- `--jobs N` / `PARALLEL_JOBS` env var for parallelism control
111+
- `--sequential` for debugging
112+
- Progress output and summary with pass/fail/skip counts
113+
- Individual test logs preserved in `.tmp/test-results/`
114+
115+
Added Makefile targets:
116+
- `test-parity-parallel`: Fast tests only (default)
117+
- `test-parity-all`: All tests including slow ones
118+
- `test-slow`: Only slow tests
119+
120+
### Results
121+
122+
- **39 fast tests** complete in ~19s with 10 parallel jobs (vs ~200s+ sequential)
123+
- **7x+ speedup** for regular CI runs
124+
- Slow tests (fuzz, stress, large-data) excluded by default but available via `--all`

zig/Makefile

Lines changed: 40 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
# make clean - Clean build artifacts
1010
# make help - Show this help
1111

12-
.PHONY: all test test-unit test-parity test-browser build clean help \
12+
.PHONY: all test test-unit test-parity test-parity-parallel test-parity-all test-slow test-browser build clean help \
1313
_test-unit _test-parity _test-browser dist \
1414
universal build-arm64 build-x64 size-report
1515

@@ -70,6 +70,38 @@ test-parity:
7070
@echo ""
7171
@cd harness && bash test-parity.sh
7272

73+
# Shell parity tests (parallel execution for faster CI)
74+
# Uses run-all-tests.sh to run all test-*.sh scripts concurrently
75+
# Default: uses nproc jobs. Override with PARALLEL_JOBS env var.
76+
# Note: Excludes slow tests by default (fuzz, stress, large-data).
77+
test-parity-parallel: build
78+
@echo ""
79+
@echo "$(CYAN)$(BOLD)════════════════════════════════════════════════════════════════$(RESET)"
80+
@echo "$(CYAN)$(BOLD) Shell Parity Tests (Parallel - Fast)$(RESET)"
81+
@echo "$(CYAN)$(BOLD)════════════════════════════════════════════════════════════════$(RESET)"
82+
@echo ""
83+
@cd harness && bash run-all-tests.sh --fast
84+
85+
# Shell parity tests including slow tests (fuzz, stress, large-data)
86+
# Use this for thorough CI runs or release validation.
87+
test-parity-all: build
88+
@echo ""
89+
@echo "$(CYAN)$(BOLD)════════════════════════════════════════════════════════════════$(RESET)"
90+
@echo "$(CYAN)$(BOLD) Shell Parity Tests (Parallel - All)$(RESET)"
91+
@echo "$(CYAN)$(BOLD)════════════════════════════════════════════════════════════════$(RESET)"
92+
@echo ""
93+
@cd harness && bash run-all-tests.sh --all
94+
95+
# Only slow tests (fuzz, stress, large-data)
96+
# Use for extended testing or CI nightly builds.
97+
test-slow: build
98+
@echo ""
99+
@echo "$(CYAN)$(BOLD)════════════════════════════════════════════════════════════════$(RESET)"
100+
@echo "$(CYAN)$(BOLD) Slow Tests Only (Fuzz, Stress, Large-Data)$(RESET)"
101+
@echo "$(CYAN)$(BOLD)════════════════════════════════════════════════════════════════$(RESET)"
102+
@echo ""
103+
@cd harness && bash run-all-tests.sh --slow
104+
73105
# Browser tests (Playwright + sql.js)
74106
test-browser:
75107
@echo ""
@@ -277,10 +309,13 @@ help:
277309
@echo "$(BOLD)Zig CR-SQLite Makefile$(RESET)"
278310
@echo ""
279311
@echo "$(BOLD)Test Targets:$(RESET)"
280-
@echo " $(CYAN)make test$(RESET) - Run all tests concurrently (unit + parity + browser)"
281-
@echo " $(CYAN)make test-unit$(RESET) - Run Zig unit tests only"
282-
@echo " $(CYAN)make test-parity$(RESET) - Run shell parity tests only"
283-
@echo " $(CYAN)make test-browser$(RESET) - Run Playwright browser tests only"
312+
@echo " $(CYAN)make test$(RESET) - Run all tests concurrently (unit + parity + browser)"
313+
@echo " $(CYAN)make test-unit$(RESET) - Run Zig unit tests only"
314+
@echo " $(CYAN)make test-parity$(RESET) - Run shell parity tests only (sequential)"
315+
@echo " $(CYAN)make test-parity-parallel$(RESET) - Run shell parity tests (parallel, excludes slow)"
316+
@echo " $(CYAN)make test-parity-all$(RESET) - Run ALL shell parity tests (including slow)"
317+
@echo " $(CYAN)make test-slow$(RESET) - Run only slow tests (fuzz, stress, large-data)"
318+
@echo " $(CYAN)make test-browser$(RESET) - Run Playwright browser tests only"
284319
@echo ""
285320
@echo "$(BOLD)Build Targets:$(RESET)"
286321
@echo " $(CYAN)make build$(RESET) - Build the Zig extension"

0 commit comments

Comments
 (0)