PNG alpha premultiplied in texture color space or linear color space?

[jimenea]

Hello,

I am trying to understand why PNG files with alpha are not giving us the expected results. The issue seems to be down to how OpenImageIO premultiplies the alpha when loading this texture. Is it done in the texture color space or the linear color space? Our results seem to indicate the former.

I have attached a single-pixel texture, cropped from the original texture, which shows this very clearly. This texture contains the pixel color (1, 1, 1, 0.5). We get that value correctly when checking with "oiiotool --dumpdata -no-autopremult", but when we take out the "-no-autopremult" option, we get "55 55 55 127 (0.21568629 0.21568629 0.21568629 0.49803925)". Notice how this value is suspiciously close to the expected value in linear space with gamma 2.2 applied (0.5 ^ 2.2 = 0.2176).

After reading https://www.realtimerendering.com/blog/a-png-puzzle/, I think we should be getting something like (187, 187, 187, 128). Am I correct? Is there anything else I should be doing? Maybe I am missing an option to change this behaviour?

Thanks in advance.

Regards,
Angel

test.png.zip

[ThiagoIze]

I know sRGB is not pure 2.2 gamma, but let's use that here because it's close enough and simpler to do. So the various values OIIO could be returning for the linear space premultiplied R channel I think are (G and B would be the same):

• (R * A)^2.2 = (1 * 0.5)^2.2 = 0.217
• (R^2.2) * A = (1^2.2) * 0.5 = 0.5

OIIO Looks to be returning 0.217, but I think the proper value is 0.5.

The 187 value Angel refers to above is I believe the value you get if you composite against black and re-apply the sRGB display transform: 0.5^(1/2.2)=0.73

So assuming OIIO is wrong, how do we handle this? As Eric's blog post points out, many image viewers do the png color space transform at the wrong point. Fixing this might break what some people expect. But we also definitely want OIIO to be doing the right thing. Do we support both with a switch? Drop support for the wrong way of doing this?

[lgritz]

I have finally gotten to the bottom of this. Stupid mistake on my part. Will have a patch in the morning.

[lgritz]

You were right that this was egregiously wrong.

It turns out we did the linearize-associate-delinearize sequence right, and it wasn't the case that we were associating in the nonlinear case.

BUT... the truth was even more embarrassing. We applied gamma, associated, and then applied 1/gamma to bring it back, when of course it should have been 1/gamma, associate, gamma.

[ThiagoIze]

lg fixed this in #4315.

[brechtvl]

I'm looking into upgrading OpenImageIO for Blender, and this change doesn't seem to be working well.

From what I can tell, the previous behavior was really the convention used by image editing tools and viewers. What I tried:

• Creating image with a gradient and saving as PNG: Pixelmator, GIMP, Blender
• Viewing: Pixelmator, GIMP, Blender, Apple Preview, Firefox

Test files: oiio_png_semi_transparent.zip. JPEG files were created using oiiotool gradient.png -o gradient_oiio.jpg.

If you for example load these in Apple Preview, and set the background to black in the Settings. Then the old OpenImageIO jpg will match the png exactly.

If you're on macOS, you can also try /Applications/Pages.app/Contents/Resources/Etched Metal.sfctex/Reflection_gray.png. OIIO seems to get this one wrong both before (too dark) and after (too bright).

I agree that in general alpha should be premultiplied in linear space. But if the convention for PNG files is to premultiply in sRGB space, and everyone follows that consistently, then actually the end result is no different than doing the correct thing?

[brechtvl]

I can create a proper issue for this, and maybe a pull request. But I'm posting here since I don't have enough context to propose a solution. The test file in this discussion is a synthetic file generated by OIIO, and the PR seems to have been motivated by something at SPI. Maybe my sample was biased and there is important software that writes PNG files like this, that is worth being compatible with, I just don't know.

[lgritz]

The details from several months back are already a little hazy in my head, but in #4315 I said:

when turning unassociated alpha into associated for gamma-corrected PNG images, we got the exponent wrong for the linearization step, and when doing the same for sRGB images, we didn't linearize at all.

I assume that the business of totally botching the true gamma-corrected images is definitely not something we should go back to, and the only part of this patch you were referring to was whether or not we linearize the sRGB data for the alpha premultiplication. It is indeed easy for me to believe that though we may now be right, doing it "wrong" is so common that we've caused more trouble than we fixed.

My recollection is that when I fixed this, yes, it also cleared up some pesky problems at SPI that I had been unable to fully understand or resolve before. So I don't want to completely revert it if there are apparently both conventions in use. I think the way forward is to allow some kind of global attribute to control whether or not we do linearize before the alpha business. Since you have a large pool of users who are likely to deal with PNG all the time, I'm happy to go with your recommendation for which seems more common and should be the default. (Whereas PNG files are a rarity for me at SPI, and I can give explicit instructions about what options to set to the people who need to handle them, so it's not really going to be hard for us if the default is the other way.)

Let me know if you have time to turn this into a PR. If not, I'm happy to give it a try myself, it's not a big deal and I guess I'm cleaning up my own mess.

[lgritz]

I have some time, I'm going to take a stab at it myself. Stay tuned.

[brechtvl]

Thanks!

We mainly need this as an option for the TextureSystem / ImageCache.

I did some testing with two images I found with gamma specified instead of sRGB. One of them the Reflection_gray.png mentioned above, the other I can't share. For those I also had to avoid the linear space transform to get the correct result. It's a small sample size though, difficult to find examples.

[lgritz]

Oh, so you think the "premultiply directly, not in linear space" needs to apply to gamma images as well? Sure, makes sense.