Alpha artifacts in fully opaque parts of textures

[notnullnotvoid]

https://cdn.discordapp.com/attachments/780432012766347315/834102101083947049/unknown.png shows what this looks like for a black texture against a bright background. Most pixels are 255 alpha, same as source image (which I double-checked does not have this problem), but there is an erratic pattern of 254 alpha pixels across the whole texture. It happens on all textures I tried and seemingly at all compression/speed levels.

[notnullnotvoid]

I should note that the asset itself isn't a solid color, the color components are multiplied to 0 in the shader. That might be relevant to whether the artifact shows up.

[frogdan]

I am also experience this issue. "Random" pixels or block have the alpha value 254 instead of 255.

Here is an image demonstrating it: https://i.imgur.com/QBz6t5M.png (the gray blocks in output alpha have the values 254).

[frogdan]

Here is some more information:

• The problems occurs regardless of build options (ie without --opt=fast-math and with only --target=sse2).
• It occurs in both a build directly from this repository, or with the updated one from bc7enc_rdo built with latest ispc.
• It occurs with both linear or perceptual error metrics.
• It occurs with completely opaque images with alpha and images without alpha.
• It occurs regardless of quality level (including ultrafast).
• It still occurs when manually disabling all alpha setting modes (pack_params.m_alpha_settings.m_use_modeX = false), so it doesn't seem to be opaque detection that's the problem.
• Manually disabling the modes with alpha in the opaque settings naturally avoids the issue.
• Further narrowing it down, at least for opaque blocks, it can be avoided by manually disabling mode6:

	pack_params.m_mode6_only = false;
	pack_params.m_opaque_settings.m_use_mode[6] = false;

This is of course with the test image I had, I can't rule out other modes behaving badly with other source data.

[frogdan]

Having looked into the details for mode6, it seems this is mostly a limitation of the format. It has a pbit on the end points which is shared between color and alpha, so for example alpha can't be 255 while color is 0. If not using the alpha (or just not for opacity), this doesn't matter. If using alpha as opacity this could be an issue. But the encoder doesn't know the intent, so can't really do that kind of judgement.

As a user, you could exclude mode6 for opaque blocks in transparent images, and make sure not to use the alpha/blending for fully opaque images. You could also ignore alpha 254/treat it as 255 in the shader.

It's also possible to modify the encoder to force pbit 1 in opaque blocks, but that also does exclude true black as mentioned above. It seems a lot simpler to just not use mode6 where it matters.

[notnullnotvoid]

Great work on the investigation! Seems like a reasonable fix would be to disable mode 6 for opaque blocks by default in the init functions, then?

[frogdan]

It might be reasonable for your case. In the general case it excludes a very useful mode.

I'd suggest just doing pack_params.m_opaque_settings.m_use_mode[6] = false; after calling the init for the case where it's a problem.

[notnullnotvoid]

For textures with transparency, it's much more important for opaque areas to actually be opaque than for them to be able to take advantage of the particular encoding features of mode 6. For textures whose alpha channel will be ignored anyway though, you'd obviously rather have the extra mode to work with. The issue is it doesn't seem that the library makes any distinction between those two use cases, presumably based on the premise that it will be able to figure out the best thing to do on a block-by-block basis, which this issue has shown to not really be the case. So to fix this without potentially hurting quality for RGB textures by default, it would be necessary to introduce an API-level distinction between RGB and RGBA textures.