Skip to content

Commit 935dc6a

Browse files
committed
vertexfilter: Adjust encodeFilterColor alpha quantization
Because the reconstruction of alpha involves replicating the low bit instead of replicating the high bit, for minimum error we should quantize the value as if it had K bits instead of K-1, and drop the low bit. This improves the maximum error by up to 40% depending on the bit count; the effect on average error is more muted and generally is around 10% or lower.
1 parent 24a4ced commit 935dc6a

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/vertexfilter.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1469,7 +1469,7 @@ void meshopt_encodeFilterColor(void* destination, size_t count, size_t stride, i
14691469
assert(unsigned((fy + fco - fcg) | (fy + fcg) | (fy - fco - fcg)) < (1u << bits));
14701470

14711471
// alpha: K-1-bit encoding with high bit set to 1
1472-
int fa = meshopt_quantizeUnorm(c[3], bits - 1) | (1 << (bits - 1));
1472+
int fa = (meshopt_quantizeUnorm(c[3], bits) >> 1) | (1 << (bits - 1));
14731473

14741474
if (stride == 4)
14751475
{

0 commit comments

Comments
 (0)