Renderer: Blending with transparent background is incorrect.

[gkjohnson]

Description

I know that blending in WebGPURenderer is now happening in Linear sRGB space but even so the result when rendering to a transparent background looks very wrong, though it's not exactly clear what's going wrong. It looks like a premultiplied alpha issue if I were to guess?

Reproduction steps

1. Set the clear color to a completely transparent value
2. Render a partially transparent object
3. Notice a different blending profile than if a solid background color were used

Code

See fiddle

Live example

https://jsfiddle.net/wj05d67x/2/

Screenshots

	bg color=#eee, alpha=0	bg color=#eee, alpha=1
WebGPURenderer
WebGLRenderer

Version

r183

Device

No response

Browser

No response

OS

No response

[Mugen87]

I actually get the same output when using WebGLRenderer + EffectComposer. In this setup, blending happens in linear-srgb as well and the color space transformation to sRGB in a separate step (OutputPass).

Demo: https://jsfiddle.net/uemry1b4/

	bg color=#eee, alpha=0, srgb output
WebGPURenderer
WebGLRenderer + EffectComposer

I can also confirm, that with linear-sRGB output color space, WebGPURenderer renders as WebGLRenderer.

https://jsfiddle.net/gzfjL7a0/1/

	bg color=#eee, alpha=0, linear-srgb output
WebGPURenderer
WebGLRenderer

[Mugen87]

What is the expected result when the canvas is blended with the HTML background and a sRGB workflow is used?

We can't use WebGLRenderer as a baseline since the inline-srgb decode produces a wrong result in the first place.

[gkjohnson]

Linear color-space or not the foreground color being blended is darker than the underlying background so it doesn't make sense that the blend result is somehow brighter than either of the colors being blended.

What is the expected result when the canvas is blended with the HTML background and a sRGB workflow is used?

The expectation is that the appearance of using a #eee solid clear color and #eee css background color with a transparent clear color should be no different.

[Mugen87]

It looks like a premultiplied alpha issue if I were to guess?

When looking at WebGLRenderer it sets the premultipliedAlpha context flag to true by default. In the spec, it is defined as follows:

If the value is true the page compositor will assume the drawing buffer contains colors with premultiplied alpha.

That means the shaders should write premultiplied alpha color values in the default framebuffer which isn't the case though. When I updated OutputPass and OutputShader accordingly, the result does still not look right: https://jsfiddle.net/noxy9cms/

[gkjohnson]

That means the shaders should write premultiplied alpha color values in the default framebuffer which isn't the case though.

Somewhat relating to #32802 & #32521 (comment), materials in three.js are always written as premultiplied alpha, so the content of any intermediate render targets always contain premultiplied alpha values unless a custom shader is being used. That means the output pass should not multiply by alpha again otherwise we will see double-darkening of colors for blending.

What I think is actually happening is that the final canvas framebuffer is expecting sRGB colors to have been multiplied by alpha but instead what we're providing is linear colors that have been premultiplied by alpha and then converted to sRGB. The linear -> sRGB brightens the colors so they no longer align the with the math in sRGB color space that the browser is expecting to do.

We can fix this by dividing linear color by the alpha to convert it back to "straight alpha", converting to sRGB, then multiplying by alpha again. This is one solution that was suggested in #32802 when using render targets as textures to account the premultiplied alpha differences compared to typical textures:

#ifdef SRGB_TRANSFER

	if ( gl_FragColor.a > 0.0 ) {

		// convert back to straight alpha, checking for zero alpha
		gl_FragColor.rgb /= gl_FragColor.a;

	}

	// convert to sRGB, the canvas output color space
	gl_FragColor = sRGBTransferOETF( gl_FragColor );

	// multiply by alpha again
	gl_FragColor.rgb *= gl_FragColor.a;

#endif

// no premultiplied alpha statement since the
// intermediate buffer is already premultiplied

[Mugen87]

Are you okay with merging the issue into #32802? They seem to be about the same topic.

[WestLangley]

That means the shaders should write premultiplied alpha color values in the default framebuffer, which isn't the case though.

It is intended that premultiplied color values are written into the default framebuffer by all built-in materials. Please advise if you have a counterexample.

What I think is actually happening is that the final canvas framebuffer is expecting sRGB colors to have been multiplied by alpha but instead what we're providing is linear colors that have been premultiplied by alpha and then converted to sRGB.

Premultiplication by alpha should only occur on linear color values. The built-in shaders in WebGLRenderer incorrectly convert to sRGB color space first. Premultiplication by alpha occurs subsequently, either in the shader itself, or as part of the blending operation. I would not use WebGLRenderer as a basis for comparison. It is also not likely to be changed at this point.

[Mugen87]

The bit that confused me was this section in the material shaders:

three.js/src/renderers/shaders/ShaderChunk/premultiplied_alpha_fragment.glsl.js

Lines 2 to 7 in 8bc275b

	#ifdef PREMULTIPLIED_ALPHA
	// Get normal blending with premultipled, use with CustomBlending, OneFactor, OneMinusSrcAlphaFactor, AddEquation.
	gl_FragColor.rgb *= gl_FragColor.a;
	#endif

The PREMULTIPLIED_ALPHA is controlled by Material.premultipliedAlpha and it defaults to false. I was under the assumption it had to be set to true when the default framebuffer expects premultiplied values. But #32521 (comment) outlined the premultiplication happens indirectly as a part of the blending function when Material.premultipliedAlpha is false.

[Mugen87]

We can fix this by dividing linear color by the alpha to convert it back to "straight alpha", converting to sRGB, then multiplying by alpha again.

It seems your suggested code fixes OutputPass. Here is a modified version of the fiddle (WebGLRendere+ EffectComposer): https://jsfiddle.net/1zsc2p3y/

[WestLangley]

@Mugen87 Premultiplication happens one way or another. It has to. Either the shader does it, or the blending function does it.