Skip to content

Commit febf550

Browse files
committed
docs: document process-level parallel linting capability
Add documentation for existing parallel linting capability using the linter binary's individual file type arguments. Changes: - Add scripts/lint-parallel.sh demonstrating parallel execution - Update linter-parallel-execution feature docs with process-level approach - Document performance comparison (15s sequential vs 14s parallel) - Explain why minimal gain (clippy dominates at ~12s/80% of time) - Add parallel execution section to linting.md with usage guidance - Recommend sequential execution for clean output in regular development Key findings: - Process-level parallelization already possible via CLI arguments - Only ~1s (7%) performance gain due to clippy domination - Trade-off: minimal speedup vs potentially interleaved output - Confirms decision to defer complex code-level implementation
1 parent a578c93 commit febf550

File tree

4 files changed

+352
-6
lines changed

4 files changed

+352
-6
lines changed

docs/contributing/linting.md

Lines changed: 36 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,24 @@ cargo run --bin linter shellcheck
7070
./scripts/linting/shellcheck.sh
7171
```
7272

73+
### Parallel Execution (Experimental)
74+
75+
For scenarios where you want to run linters concurrently:
76+
77+
```bash
78+
# Run linters in parallel using process-level parallelization
79+
./scripts/lint-parallel.sh
80+
```
81+
82+
**Note**: Parallel execution provides minimal performance improvement (~1s, 7% faster) and may produce interleaved output. Sequential execution is recommended for regular development.
83+
84+
**When to use**:
85+
86+
- ✅ CI/CD pipelines where every second counts
87+
- ❌ Regular development (use sequential for clean output)
88+
89+
See [Linter Parallel Execution Feature](../features/linter-parallel-execution/README.md) for detailed analysis and trade-offs.
90+
7391
## 📋 Tool-Specific Guidelines
7492

7593
### Markdown Linting (`markdownlint-cli`)
@@ -352,11 +370,26 @@ rustup component add clippy rustfmt
352370

353371
```bash
354372
# Run specific linters for faster feedback during development
355-
cargo run --bin linter markdown # Only markdown (fastest)
356-
cargo run --bin linter yaml # Only YAML files
357-
cargo run --bin linter clippy # Only Rust analysis (slowest)
373+
cargo run --bin linter markdown # Only markdown (~1s)
374+
cargo run --bin linter yaml # Only YAML files (~0.2s)
375+
cargo run --bin linter clippy # Only Rust analysis (~12s - slowest)
376+
377+
# Run non-Rust linters for quick checks
378+
cargo run --bin linter markdown
379+
cargo run --bin linter yaml
380+
cargo run --bin linter toml
381+
# Skip clippy for faster iteration during active development
358382
```
359383

384+
**Parallel execution** is also possible but provides minimal benefit:
385+
386+
```bash
387+
# Process-level parallelization (experimental, ~1s faster)
388+
./scripts/lint-parallel.sh
389+
```
390+
391+
Note: Parallel execution trades clean output for minimal speed gain. Use sequential execution for regular development.
392+
360393
## 🚨 Troubleshooting
361394

362395
### Common Issues

docs/features/linter-parallel-execution/README.md

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,46 @@ Clarifying questions for implementation (if feature is prioritized in the future
3636

3737
**Status**: **Deferred** - Not a priority for initial implementation
3838

39-
**Reason**: Current performance (13s) is acceptable for pre-commit workflow. Implementation complexity outweighs performance gains.
39+
**Reason**: Current performance (15s) is acceptable for pre-commit workflow. Implementation complexity outweighs minimal performance gains.
40+
41+
**Reconsider when**: Execution time exceeds **25 seconds** (more linters added) or auto-fix feature makes it too slow.
42+
43+
## ✅ Alternative: Process-Level Parallelization (Already Possible)
44+
45+
**Discovery**: The linter binary already supports running individual linter types via command-line arguments, which enables **process-level parallelization** without code changes.
46+
47+
**Script Location**: `scripts/lint-parallel.sh`
48+
49+
**How it works**:
50+
51+
```bash
52+
# Run individual linters in parallel processes
53+
./target/release/linter markdown &
54+
./target/release/linter yaml &
55+
./target/release/linter toml &
56+
./target/release/linter shellcheck &
57+
./target/release/linter rustfmt &
58+
wait
59+
60+
# Run clippy sequentially (may conflict with rustfmt)
61+
./target/release/linter clippy
62+
63+
# Run cspell separately (read-only)
64+
./target/release/linter cspell
65+
```
66+
67+
**Performance**: ~14s (vs 15s sequential) - minimal improvement due to clippy dominating execution time (~12s)
68+
69+
**Trade-offs**:
70+
71+
- ✅ No code changes required
72+
- ✅ Simple shell script implementation
73+
- ✅ Easy to adjust grouping strategy
74+
- ❌ Output may be interleaved (less readable)
75+
- ❌ Minimal performance gain (~1s) since clippy dominates
76+
- ❌ Less control over error aggregation and reporting
77+
78+
**Recommendation**: Use sequential execution (`cargo run --bin linter all`) for clean output. Process-level parallelization provides minimal benefit and potentially confusing output.
4079

4180
**Reconsider when**: Execution time exceeds **25 seconds** (more linters added) or auto-fix feature makes it too slow.
4281

docs/features/linter-parallel-execution/specification.md

Lines changed: 121 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -358,11 +358,130 @@ If/when this feature is implemented:
358358
- [Development Principles](../../development-principles.md)
359359
- [Linting Guide](../../contributing/linting.md)
360360

361-
## 📊 Priority
361+
## � Alternative Approach: Process-Level Parallelization
362+
363+
### Discovery: Existing CLI Support
364+
365+
The linter binary already supports running individual linter types via command-line arguments:
366+
367+
```bash
368+
cargo run --bin linter markdown
369+
cargo run --bin linter yaml
370+
cargo run --bin linter toml
371+
cargo run --bin linter clippy
372+
cargo run --bin linter rustfmt
373+
cargo run --bin linter shellcheck
374+
cargo run --bin linter cspell
375+
```
376+
377+
This enables **process-level parallelization** without any code changes - simply run multiple linter processes concurrently using shell job control.
378+
379+
### Implementation: Shell Script
380+
381+
**Location**: `scripts/lint-parallel.sh`
382+
383+
**Approach**:
384+
385+
```bash
386+
#!/bin/bash
387+
388+
# Build once in release mode for better performance
389+
cargo build --release --bin linter --quiet
390+
LINTER_BIN="./target/release/linter"
391+
392+
# Group 1: Run linters in parallel (different file types)
393+
"$LINTER_BIN" markdown &
394+
"$LINTER_BIN" yaml &
395+
"$LINTER_BIN" toml &
396+
"$LINTER_BIN" shellcheck &
397+
"$LINTER_BIN" rustfmt &
398+
wait
399+
400+
# Group 2: Run clippy sequentially
401+
"$LINTER_BIN" clippy
402+
403+
# Separate: Run cspell (read-only)
404+
"$LINTER_BIN" cspell
405+
```
406+
407+
### Performance Comparison
408+
409+
**Sequential execution** (`cargo run --bin linter all`):
410+
411+
- Total time: ~15 seconds
412+
- Output: Clean, grouped by linter
413+
- All errors displayed in logical order
414+
415+
**Process-level parallel execution** (`./scripts/lint-parallel.sh`):
416+
417+
- Total time: ~14 seconds (7% faster)
418+
- Output: May be interleaved from concurrent processes
419+
- Limited improvement because clippy dominates (~12s out of 15s)
420+
421+
### Why Minimal Performance Gain?
422+
423+
**Execution time breakdown**:
424+
425+
- clippy: ~12s (80% of total time) - runs sequentially
426+
- markdown: ~1s
427+
- yaml: ~0.15s
428+
- toml: ~0.07s
429+
- rustfmt: ~0.2s
430+
- shellcheck: ~0.03s
431+
- cspell: ~1.6s
432+
433+
**Analysis**: Clippy dominates execution time, so parallelizing the other fast linters (~3s combined) only saves ~1 second.
434+
435+
**Theoretical maximum speedup**: Even if all non-clippy linters ran instantly, total time would be ~12s (clippy) + ~0s (others) = ~12s, only ~3s improvement from current 15s.
436+
437+
### Trade-offs: Process-Level vs Code-Level Parallelization
438+
439+
| Aspect | Process-Level (Shell Script) | Code-Level (Async Refactor) |
440+
| -------------------- | ---------------------------- | ------------------------------------- |
441+
| **Implementation** | ✅ Simple shell script | ❌ Complex async refactoring |
442+
| **Code changes** | ✅ None required | ❌ All 7 linters need refactoring |
443+
| **Performance gain** | ⚠️ Minimal (~1s, 7%) | ⚠️ Similar (~1-2s at best) |
444+
| **Output quality** | ❌ May be interleaved | ✅ Clean, sequential display |
445+
| **Error handling** | ❌ Basic process exit codes | ✅ Rich error aggregation |
446+
| **Maintenance** | ✅ Easy to modify | ❌ More complex to maintain |
447+
| **Testing** | ✅ Simple to test | ❌ Requires async test infrastructure |
448+
| **Dependencies** | ✅ None | ❌ Adds tokio/async runtime |
449+
450+
### Recommendation
451+
452+
**Use sequential execution** (`cargo run --bin linter all`) because:
453+
454+
1. **Clean output**: Errors are grouped by linter and easy to read
455+
2. **Minimal speedup**: Process-level parallelization only saves ~1s (7%)
456+
3. **Simplicity**: No additional scripts or complexity needed
457+
4. **Maintenance**: One less thing to maintain
458+
459+
**When to use process-level parallelization**:
460+
461+
- Never recommended for regular development workflow
462+
- Could be useful for CI/CD if every second counts (but 1s is negligible)
463+
- Better to wait for more linters to be added (if ever) before optimizing
464+
465+
**When to implement code-level parallelization**:
466+
467+
- Execution time exceeds 25 seconds (more linters added)
468+
- Clippy execution time is significantly reduced
469+
- Auto-fix feature makes linting much slower
470+
471+
### Conclusion
472+
473+
The discovery that process-level parallelization is already possible **confirms the initial decision** to defer implementation:
474+
475+
- ✅ Minimal performance gain even with perfect parallelization
476+
- ✅ Clean sequential output is more valuable than 1s speedup
477+
- ✅ No compelling reason to add complexity
478+
- ✅ YAGNI principle applies - implement only if truly needed
479+
480+
## �📊 Priority
362481

363482
**Priority**: Low (Future Enhancement)
364483

365-
**Reason**: Current performance is acceptable. Focus on higher-value features first (like auto-fix).
484+
**Reason**: Current performance (15s) is acceptable. Process-level parallelization available but provides minimal benefit (1s). Focus on higher-value features first (like auto-fix).
366485

367486
**Decision**: Defer implementation until there's clear evidence it's needed.
368487

scripts/lint-parallel.sh

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
#!/bin/bash
2+
3+
# Parallel Linting Script
4+
#
5+
# This script demonstrates running linters in parallel by calling the linter binary
6+
# with individual file type arguments. Each linter runs in a separate process.
7+
#
8+
# Usage: ./scripts/lint-parallel.sh
9+
#
10+
# This approach provides parallel execution without modifying the linter binary,
11+
# but has trade-offs:
12+
#
13+
# Pros:
14+
# - No code changes required - uses existing CLI interface
15+
# - Simple to implement and understand
16+
# - Real parallel execution with performance gains
17+
# - Easy to adjust grouping strategy
18+
#
19+
# Cons:
20+
# - Output may be interleaved (mixed messages from different linters)
21+
# - Multiple Rust compilation/startup overhead if binary not cached
22+
# - Less control over output formatting
23+
# - Harder to aggregate errors cleanly
24+
25+
set -e
26+
27+
echo "========================================"
28+
echo "Running Linters in Parallel"
29+
echo "========================================"
30+
echo ""
31+
32+
# Build the linter binary once (release mode for better performance)
33+
echo "Building linter binary..."
34+
cargo build --release --bin linter --quiet
35+
LINTER_BIN="./target/release/linter"
36+
37+
echo ""
38+
39+
# Track overall success/failure
40+
FAILED=0
41+
42+
# Temporary directory for storing results
43+
RESULT_DIR=$(mktemp -d)
44+
trap 'rm -rf "$RESULT_DIR"' EXIT
45+
46+
# Function to run a linter and capture its exit code
47+
run_linter() {
48+
local linter_name=$1
49+
local result_file="$RESULT_DIR/$linter_name.result"
50+
51+
echo "Starting $linter_name linter..."
52+
53+
if "$LINTER_BIN" "$linter_name" > "$result_file.log" 2>&1; then
54+
echo "0" > "$result_file"
55+
echo "$linter_name completed successfully"
56+
else
57+
echo "1" > "$result_file"
58+
echo "$linter_name failed"
59+
fi
60+
}
61+
62+
# Start timing
63+
START_TIME=$(date +%s)
64+
65+
# Group 1: Run linters in parallel (different file types - no conflicts)
66+
echo "Group 1: Running linters in parallel (markdown, yaml, toml, shellcheck, rustfmt)..."
67+
echo ""
68+
69+
run_linter "markdown" &
70+
PID_MARKDOWN=$!
71+
72+
run_linter "yaml" &
73+
PID_YAML=$!
74+
75+
run_linter "toml" &
76+
PID_TOML=$!
77+
78+
run_linter "shellcheck" &
79+
PID_SHELLCHECK=$!
80+
81+
run_linter "rustfmt" &
82+
PID_RUSTFMT=$!
83+
84+
# Wait for Group 1 to complete
85+
wait $PID_MARKDOWN $PID_YAML $PID_TOML $PID_SHELLCHECK $PID_RUSTFMT
86+
87+
echo ""
88+
echo "Group 1 completed."
89+
echo ""
90+
91+
# Group 2: Run clippy sequentially (may conflict with rustfmt on .rs files)
92+
echo "Group 2: Running clippy (sequential)..."
93+
echo ""
94+
95+
run_linter "clippy"
96+
97+
echo ""
98+
echo "Group 2 completed."
99+
echo ""
100+
101+
# Separate group: Run cspell (checks all files, read-only)
102+
echo "Running cspell (read-only checker)..."
103+
echo ""
104+
105+
run_linter "cspell"
106+
107+
echo ""
108+
109+
# Calculate elapsed time
110+
END_TIME=$(date +%s)
111+
ELAPSED=$((END_TIME - START_TIME))
112+
113+
# Display results
114+
echo ""
115+
echo "========================================"
116+
echo "Linting Results"
117+
echo "========================================"
118+
echo ""
119+
120+
for linter in markdown yaml toml shellcheck rustfmt clippy cspell; do
121+
result_file="$RESULT_DIR/$linter.result"
122+
log_file="$RESULT_DIR/$linter.result.log"
123+
124+
if [ -f "$result_file" ]; then
125+
exit_code=$(cat "$result_file")
126+
if [ "$exit_code" -eq 0 ]; then
127+
echo "$linter: PASSED"
128+
else
129+
echo "$linter: FAILED"
130+
FAILED=1
131+
132+
# Show the error log
133+
if [ -f "$log_file" ]; then
134+
echo " Log:"
135+
sed 's/^/ /' "$log_file"
136+
fi
137+
fi
138+
fi
139+
done
140+
141+
echo ""
142+
echo "========================================"
143+
echo "Total time: ${ELAPSED}s"
144+
echo "========================================"
145+
146+
# Exit with error if any linter failed
147+
if [ $FAILED -eq 1 ]; then
148+
echo ""
149+
echo "❌ Some linters failed. Please fix the issues above."
150+
exit 1
151+
else
152+
echo ""
153+
echo "✅ All linters passed!"
154+
exit 0
155+
fi

0 commit comments

Comments
 (0)