Fix severe boundary artifacts in Box filter caused by floating-point precision#255
Open
DavidDiao wants to merge 1 commit intonodeca:masterfrom
Open
Fix severe boundary artifacts in Box filter caused by floating-point precision#255DavidDiao wants to merge 1 commit intonodeca:masterfrom
DavidDiao wants to merge 1 commit intonodeca:masterfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes a severe accuracy bug specifically affecting the Box filter during downsampling.
Users migrating from or comparing outputs with
PIL/Pillowmight notice that at certain downsampling ratios (e.g.,1000px -> 288px),picaproduces periodic visual glitches.The Root Cause
Unlike continuous filters (Lanczos, Bicubic) which gracefully decay to
0.0at their window edges, the Box filter is a discrete step function. It relies on a hard cliff: a pixel is either fully included (weight1.0) or completely excluded (weight0.0).When calculating the normalized distance of a boundary pixel to the center
diff = (pxl + 0.5) - center:0.5or-0.5.-0.49999999999999994.Math.abs(x) < 0.5evaluates this astrue.As a result, an out-of-bounds pixel that should have been discarded (weight
0.0) incorrectly receives a full weight of1.0. This accidentally widens the averaging window by 1 extra pixel for certain periodic target pixels, drastically changing the denominator and introducing a neighboring color, causing the massive deviation.The Fix
This PR implements a mathematically robust boundary extraction specifically to defend against floating-point jitter at exactly
±0.5for the Box filter. By offsetting theMath.floorbounds by+0.5(a standard technique used in PIL's C implementation), we physically exclude the floating-point-corrupted pixels from entering the accumulation loop.Impact
0.0atx = support, floating-point boundary jitter does not impact their sum.