Skip to content

Commit 2bd6309

Browse files
claudeclaude
committed
Add GitHub Action with automatic caching
New action.yml provides a simple interface for GitHub Actions: - Split: distributes tests across workers with index output - Merge: saves timings from JUnit XML files - Automatic cache restore/save - no manual setup needed Usage: uses: dashdoc/fairsplice@v1 with: command: split pattern: 'tests/**/*.py' total: 3 index: ${{ matrix.index }}
1 parent 7a2e5a5 commit 2bd6309

File tree

2 files changed

+179
-99
lines changed

2 files changed

+179
-99
lines changed

‎README.md‎

Lines changed: 91 additions & 99 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,52 @@
11
# Fairsplice
22

3-
**Warning: this project is still in very early development!**
3+
Fairsplice is a CLI tool and GitHub Action that optimizes test distribution across parallel workers. It provides CircleCI-style test splitting based on timing data for GitHub Actions.
44

5-
Fairsplice is a CLI tool designed to optimize test distribution across multiple workers. By intelligently splitting and saving test cases, Fairsplice ensures a balanced workload distribution for your CI/CD pipelines, making tests run time more predictable.
5+
## Quick Start (GitHub Action)
66

7-
We found Github Actions lacking when compared to CircleCI which has [tests splitting](https://circleci.com/docs/parallelism-faster-jobs/#how-test-splitting-works) based on timings.
8-
9-
There are a number of projects like [Split tests](https://github.com/marketplace/actions/split-tests) but they require uploading and downloading Junit XML files and merging them, or committing the Junit files to have them when running the tests.
7+
```yaml
8+
jobs:
9+
test:
10+
runs-on: ubuntu-latest
11+
strategy:
12+
matrix:
13+
index: [0, 1, 2]
14+
steps:
15+
- uses: actions/checkout@v4
16+
17+
- name: Split tests
18+
id: split
19+
uses: dashdoc/fairsplice@v1
20+
with:
21+
command: split
22+
pattern: 'tests/**/*.py'
23+
total: 3
24+
index: ${{ matrix.index }}
25+
26+
- name: Run tests
27+
run: pytest ${{ steps.split.outputs.tests }} --junit-xml=junit-${{ matrix.index }}.xml
28+
29+
- uses: actions/upload-artifact@v4
30+
with:
31+
name: junit-${{ matrix.index }}
32+
path: junit-${{ matrix.index }}.xml
33+
34+
save-timings:
35+
needs: test
36+
runs-on: ubuntu-latest
37+
steps:
38+
- uses: actions/checkout@v4
39+
40+
- uses: actions/download-artifact@v4
41+
42+
- name: Merge timings
43+
uses: dashdoc/fairsplice@v1
44+
with:
45+
command: merge
46+
prefix: 'junit-*/junit-'
47+
```
1048
11-
This tool stores test timings in a local JSON file, keeping the last 10 timings for each test file and using the average for splitting. No external database required!
49+
That's it! Caching is handled automatically.
1250
1351
## How It Works
1452
@@ -21,7 +59,7 @@ This tool stores test timings in a local JSON file, keeping the last 10 timings
2159
│ 1. SPLIT PHASE │
2260
└─────────────────────┘
2361

24-
timings.json fairsplice split
62+
timings (cached) fairsplice split
2563
┌──────────────────────┐ ┌─────────────────┐
2664
│ { │ │ │
2765
│ "test_a.py": [2.1],│ ──────▶ │ Load timings │
@@ -39,10 +77,9 @@ This tool stores test timings in a local JSON file, keeping the last 10 timings
3977
▼ ▼ ▼
4078
┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐
4179
│ Worker 0 │ │ Worker 1 │ │ Worker 2 │
42-
│ ["test_b.py"] │ │ ["test_a.py", │ │ ["test_c.py"] │
43-
│ ~5.3s │ │ "test_c.py"] │ │ ~1.8s │
44-
└─────────┬─────────┘ │ ~3.9s │ └─────────┬─────────┘
45-
│ └─────────┬─────────┘ │
80+
│ ~5.3s │ │ ~3.9s │ │ ~5.1s │
81+
└─────────┬─────────┘ └─────────┬─────────┘ └─────────┬─────────┘
82+
│ │ │
4683
▼ ▼ ▼
4784
┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐
4885
│ Run tests │ │ Run tests │ │ Run tests │
@@ -57,130 +94,85 @@ This tool stores test timings in a local JSON file, keeping the last 10 timings
5794
▼
5895
┌─────────────────────────┐
5996
│ fairsplice merge │
60-
│ --prefix junit- │
61-
└─────────────────────────┘
62-
│
63-
▼
64-
┌─────────────────────────┐
65-
│ Extract timings from │
66-
│ JUnit XML results │
97+
│ (extracts timings) │
6798
└─────────────────────────┘
6899
│
69100
▼
70101
┌──────────────────────┐
71-
│ timings.json │
72-
│ Updated with new │
73-
│ timing data │◀─── Cached/committed
102+
│ timings (cached) │◀─── Auto-cached
74103
└──────────────────────┘ for next run
75104
```
76105

77106
**Key concepts:**
78-
- **Split phase**: Before tests run, fairsplice distributes test files across workers based on historical timing data
79-
- **Merge phase**: After tests complete, fairsplice extracts timing from JUnit XML and updates the timings file
80-
- **Bin packing**: Tests are assigned to workers to balance total execution time (heaviest tests first)
81-
- **Rolling average**: Keeps last 10 timings per test file, uses average for predictions
107+
- **Split phase**: Distributes test files across workers based on historical timing data
108+
- **Merge phase**: Extracts timing from JUnit XML and caches for next run
109+
- **Bin packing**: Assigns tests to balance total execution time (heaviest tests first)
110+
- **Rolling average**: Keeps last 10 timings per test file for predictions
82111

83-
## Installation
112+
## GitHub Action Reference
84113

85-
This project is built using [Bun](https://bun.sh).
114+
### Inputs
86115

87-
Ensure you have Bun installed.
88-
To launch it, run
116+
| Input | Required | Description |
117+
|-------|----------|-------------|
118+
| `command` | Yes | `split` or `merge` |
119+
| `timings-file` | No | JSON file for timings (default: `.fairsplice-timings.json`) |
120+
| `pattern` | For split | Glob pattern to match test files |
121+
| `total` | For split | Total number of workers |
122+
| `index` | For split | Current worker index (0-based) |
123+
| `prefix` | For merge | Prefix to match JUnit XML files |
89124

90-
```bash
91-
bunx fairsplice
92-
```
93-
94-
## Usage
125+
### Outputs
95126

96-
Fairsplice has two commands: `merge` and `split`. Both require a `--timings-file` parameter.
127+
| Output | Description |
128+
|--------|-------------|
129+
| `tests` | Space-separated list of test files (when `index` provided) |
130+
| `buckets` | JSON array of all test buckets |
97131

98-
### Merging test results
132+
## CLI Usage
99133

100-
Save test timings from JUnit XML file(s):
134+
Install with Bun:
101135

102136
```bash
103-
fairsplice merge --timings-file <timings.json> --prefix <prefix>
104-
```
105-
106-
- `--timings-file <file>`: JSON file to store timings
107-
- `--prefix <prefix>`: Prefix to match JUnit XML files
108-
109-
Example:
110-
111-
```bash
112-
# Merges junit-0.xml, junit-1.xml, junit-2.xml, etc.
113-
fairsplice merge --timings-file timings.json --prefix junit-
137+
bunx fairsplice
114138
```
115139

116-
### Splitting test cases
117-
118-
Split test files across workers based on historical timings:
140+
### Commands
119141

142+
**Split tests:**
120143
```bash
121-
fairsplice split --timings-file <timings.json> --pattern "<pattern>" --total <total> --out <file>
144+
fairsplice split --timings-file timings.json --pattern "tests/**/*.py" --total 3 --out split.json
122145
```
123146

124-
- `--timings-file <file>`: JSON file with stored timings
125-
- `--pattern "<pattern>"`: Pattern to match test files (can be used multiple times)
126-
- `--total <total>`: Total number of workers
127-
- `--out <file>`: File to write split result to (JSON array of arrays)
128-
- `--replace-from <string>`: (Optional) Substring to replace in file paths
129-
- `--replace-to <string>`: (Optional) Replacement string
130-
131-
Example:
132-
147+
**Merge results:**
133148
```bash
134-
fairsplice split --timings-file timings.json --pattern "test_*.py" --total 3 --out split.json
149+
fairsplice merge --timings-file timings.json --prefix junit-
135150
```
136151

137-
## Using with GitHub Actions
152+
### CLI Options
138153

139-
To persist timings across CI runs, use GitHub Actions cache:
140-
141-
```yaml
142-
- name: Cache test timings
143-
uses: actions/cache@v4
144-
with:
145-
path: timings.json
146-
key: fairsplice-timings-${{ github.ref }}
147-
restore-keys: |
148-
fairsplice-timings-
149-
150-
- name: Split tests
151-
run: bunx fairsplice split --timings-file timings.json --pattern "tests/**/*.py" --total 3 --out split.json
152154
```
153-
154-
Alternatively, you can commit the timings file to your repository for simpler persistence.
155-
156-
## Help
157-
158-
For a detailed list of commands and options, use the help command:
159-
160-
```bash
161-
fairsplice --help
155+
fairsplice split
156+
--timings-file <file> JSON file with stored timings
157+
--pattern <pattern> Glob pattern for test files (can repeat)
158+
--total <n> Number of workers
159+
--out <file> Output JSON file
160+
161+
fairsplice merge
162+
--timings-file <file> JSON file to store timings
163+
--prefix <prefix> Prefix to match JUnit XML files
162164
```
163165

164166
## Contributing
165167

166-
Contributions are welcome! Please fork the repository and submit a pull request with your improvements.
167-
168-
### Running locally
169-
170-
Launch the development version with:
171-
172168
```bash
169+
# Run locally
173170
bun run index.ts
174-
```
175-
176-
### Running tests
177171

178-
Launch the following command to run tests:
179-
180-
```bash
181-
bun test [--watch]
172+
# Run tests
173+
bun test
182174
```
183175

184176
## License
185177

186-
Fairsplice is open-source software licensed under the MIT license.
178+
MIT

‎action.yml‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
name: 'Fairsplice'
2+
description: 'Split tests across parallel workers based on timing data'
3+
branding:
4+
icon: 'scissors'
5+
color: 'blue'
6+
7+
inputs:
8+
command:
9+
description: 'Command to run: split or merge'
10+
required: true
11+
timings-file:
12+
description: 'JSON file to store/read timings (default: .fairsplice-timings.json)'
13+
required: false
14+
default: '.fairsplice-timings.json'
15+
# split inputs
16+
pattern:
17+
description: 'Glob pattern to match test files (for split)'
18+
required: false
19+
total:
20+
description: 'Total number of workers (for split)'
21+
required: false
22+
index:
23+
description: 'Current worker index, 0-based (for split) - outputs only this worker tests'
24+
required: false
25+
# merge inputs
26+
prefix:
27+
description: 'Prefix to match JUnit XML files (for merge)'
28+
required: false
29+
30+
outputs:
31+
tests:
32+
description: 'Space-separated list of test files for the current worker (when index is provided)'
33+
value: ${{ steps.split.outputs.tests }}
34+
buckets:
35+
description: 'JSON array of test buckets (when index is not provided)'
36+
value: ${{ steps.split.outputs.buckets }}
37+
38+
runs:
39+
using: 'composite'
40+
steps:
41+
- name: Setup Bun
42+
uses: oven-sh/setup-bun@v2
43+
44+
- name: Restore timings cache
45+
uses: actions/cache/restore@v4
46+
with:
47+
path: ${{ inputs.timings-file }}
48+
key: fairsplice-timings-${{ github.repository }}-${{ github.ref_name }}
49+
restore-keys: |
50+
fairsplice-timings-${{ github.repository }}-
51+
52+
- name: Run split
53+
id: split
54+
if: inputs.command == 'split'
55+
shell: bash
56+
run: |
57+
# Run fairsplice split
58+
bunx fairsplice@latest split \
59+
--timings-file "${{ inputs.timings-file }}" \
60+
--pattern "${{ inputs.pattern }}" \
61+
--total "${{ inputs.total }}" \
62+
--out /tmp/fairsplice-buckets.json
63+
64+
# Output results
65+
if [ -n "${{ inputs.index }}" ]; then
66+
# Extract tests for specific worker index
67+
TESTS=$(jq -r '.[${{ inputs.index }}] | join(" ")' /tmp/fairsplice-buckets.json)
68+
echo "tests=$TESTS" >> $GITHUB_OUTPUT
69+
else
70+
# Output all buckets
71+
BUCKETS=$(cat /tmp/fairsplice-buckets.json)
72+
echo "buckets=$BUCKETS" >> $GITHUB_OUTPUT
73+
fi
74+
75+
- name: Run merge
76+
if: inputs.command == 'merge'
77+
shell: bash
78+
run: |
79+
bunx fairsplice@latest merge \
80+
--timings-file "${{ inputs.timings-file }}" \
81+
--prefix "${{ inputs.prefix }}"
82+
83+
- name: Save timings cache
84+
if: inputs.command == 'merge'
85+
uses: actions/cache/save@v4
86+
with:
87+
path: ${{ inputs.timings-file }}
88+
key: fairsplice-timings-${{ github.repository }}-${{ github.ref_name }}-${{ github.run_id }}

0 commit comments

Comments
 (0)