perf: add benchmark suite with CI comparison#10
Merged
Conversation
- mitata benchmarks for per-feature and worst-case scenarios - Welch's t-test (via jstat) for statistical significance - CI compares PR benchmarks against main, flags regressions - Move logos to static/logos/, docs images to static/docs/
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Benchmark Comparison: main vs perf/add-benchmarksThreshold: 5%+ change, >100us absolute delta, and statistically significant (p<0.05).
No regressions detected. react-logo-soup Benchmark ReportTest fixtures: 63 real SVGs from static/logos/. 2000ms budget per bench, 30-2000 samples. Feature Comparisons (Welch's t-test)
A/B columns match the order in the test name. Sig: Full benchmark output in the CI job logs. |
…ut dir - Drop mitata, jstat, @resvg/resvg-js (3 deps removed) - Add @napi-rs/canvas for real Skia-backed pixel rendering - Load all 63 SVGs from static/logos/ — real measurements, no synthetics - Inline Welch's t-test (no jstat) — self-contained welch.ts - Single collectSamples timing loop (was duplicated with mitata) - All bench output goes to tmp/ (single gitignore entry) - CI: only yaml writes GITHUB_STEP_SUMMARY, simplified job URL - Seed PRNG removed (no longer needed — fixtures are deterministic from real SVGs)
Key benchmarks: - measure: single logo pipeline with median-sized real logo (not thin outlier) - getVCT x 20: per-render cost with real normalized logos - mount 20 logos: real-world scenario — 20 different logos, default settings A/B comparisons: - density ON vs OFF: quantifies densityAware cost - visual-center-y vs bounds: render-path alignment cost - bbox scaling: large vs small resolution impact Dropped: sub-100ns math benchmarks (calcDims, createNormalizedLogo), worstCase 1 (≈ fullPipeline), linear 100-vs-20 comparison, calcDims density A/B (always 'not significant'), normalized100 fixture.
- densityAware: true vs false - alignBy: visual-center-y vs bounds - cropToContent: true vs false Replaced bbox scaling test (internal detail, unrealistic sizes) with cropToContent (real user option). Removed unused bboxSmall/ bboxLarge fixtures and detectContentBoundingBox import.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds per-feature and worst-case benchmarks using mitata, with Welch's t-test for statistical significance (via jstat). On PRs, CI runs benchmarks on both the PR and main, then compares and flags regressions.
Also consolidates static assets: logos to
static/logos/, doc images tostatic/docs/.