Automate manual audio regression comparisons#1758
Open
derselbst wants to merge 16 commits intoFluidSynth:masterfrom
Open
Automate manual audio regression comparisons#1758derselbst wants to merge 16 commits intoFluidSynth:masterfrom
derselbst wants to merge 16 commits intoFluidSynth:masterfrom
Conversation
Co-authored-by: derselbst <8152480+derselbst@users.noreply.github.com>
Co-authored-by: derselbst <8152480+derselbst@users.noreply.github.com>
Co-authored-by: derselbst <8152480+derselbst@users.noreply.github.com>
Co-authored-by: derselbst <8152480+derselbst@users.noreply.github.com>
Co-authored-by: derselbst <8152480+derselbst@users.noreply.github.com>
Co-authored-by: derselbst <8152480+derselbst@users.noreply.github.com>
Member
Author
|
@spessasus FYI |
Member
Author
|
Why did I close this? |
Contributor
|
I have no idea! The PR looks good so far. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The manual rendering tests previously required human inspection and could not be executed in CI. Inspired by the discussion in #1747, I fired up copilot to introduce an automated workflow that compiles a reference FluidSynth revision, renders the audio with the reference and current version of FluidSynth, and compares SNR/RMS/ABS metrics, without requiring versioned "golden reference" files.
check_renderingto run against a caller-specifiedfluidsynthbinary viaMANUAL_TEST_FLUIDSYNTH.test/run-manual-regression.shto build current + reference revisions, render both sets, and compare with SoX metrics and thresholds.rendering-regression.ymlto run the regression comparison with configurable reference ref and thresholds.Example usage:
TODO:
-o synth.limiter.active=1