Remove Redis dependency, use local JSON file for timing storage#1
Closed
Remove Redis dependency, use local JSON file for timing storage#1
Conversation
Replace external Redis database with a simple local JSON file storage: - Add src/backend/fileStorage.ts with same saveTimings/getTimings interface - Remove src/backend/redis.ts and redis package dependency - Update config.ts to use FAIRSPLICE_TIMINGS_FILE env var (default: .fairsplice-timings.json) - Update commands to use new file-based storage - Update README with new configuration and GitHub Actions cache example This eliminates the need for any external database while maintaining the same functionality (storing last 10 timings per test file).
- Add GitHub Actions workflow (test.yml) with three jobs: - unit-tests: runs existing unit tests in src/ - dummy-tests: runs dummy tests, saves timings, and splits for workers - fairsplice-integration: matrix job testing split across 3 workers - Add dummy test files with varying execution times: - fast.test.ts: ~100ms total (50ms, 30ms, 20ms delays) - medium.test.ts: ~450ms total (150ms, 200ms, 100ms delays) - slow.test.ts: ~900ms total (300ms, 350ms, 250ms delays) - variable.test.ts: ~600ms total (25ms, 175ms, 400ms delays) - Add fileStorage.test.ts to test the JSON file storage backend
Add ASCII diagram explaining the two-phase workflow: 1. Split phase: load timings, distribute tests via bin packing 2. Save phase: extract timings from JUnit XML, update storage Includes key concepts section explaining bin packing and rolling averages.
The integration test was incorrectly running all tests in each worker before splitting. Now it properly simulates a real CI workflow: 1. integration-prepare: Restores cached timings (or generates once on first run), splits tests, uploads split config as artifact 2. integration-worker (x3): Downloads split config, runs ONLY assigned tests, uploads JUnit results 3. integration-finalize: Collects all worker results, saves updated timings to cache for next run This demonstrates how fairsplice is intended to be used: - Timings persist across CI runs via cache - Tests run only once (not duplicated across workers) - Each worker runs its fair share based on historical timings
Remove the step that ran all tests to generate initial timings - this defeats the purpose of test splitting in a real scenario where the full suite could take hours. Now the workflow correctly handles cold starts: - First run: No cached timings, fairsplice uses default timing (10s) per test, tests are split evenly across workers - After run: Timings are saved from worker results - Subsequent runs: Cached timings enable optimal distribution This matches real-world usage where you can't afford to run the full test suite just to gather timing data.
Workers only need to: 1. Download split.json artifact 2. Parse JSON to get test list (using jq) 3. Run their test framework Changes: - Remove 'bun install' from workers (not needed) - Use jq for JSON parsing (cleaner than inline bun script) - Add comments clarifying workers don't need fairsplice - Use GitHub outputs for cleaner conditional flow
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace external Redis database with a simple local JSON file storage:
This eliminates the need for any external database while maintaining
the same functionality (storing last 10 timings per test file).