Color Management & HDR

[sparklescrewdriver]

Describe the feature

There has been no progress on implementing HDR within the X11 protocol, though it could potentially be approached in two ways. One option is to follow Wayland’s lead (in terms of code i.e. copy it and adjust any licensing requirements to make this feasible) and transition to in sort a X11.1 protocol. This path would be simpler and require less effort but could introduce challenges, such as increased dependency on the Wayland project and reduced independence. It’s a discussion that needs to be revisited, it was considered once before, about five years ago. The topic has become more relevant now, as Wayland has HDR support. However, it might be more beneficial to focus on this improvement after the codebase has been cleaned up. There could also be the introduction of a deep color expansion to the colormap sections of the project. To start off with there would need to the development of a check list of the steps to do - initially when Wayland started the process it looked like the following (sharing to give ideas):

• improve sharing of ICC files !240improve sharing of ICC files !240 (merged)
• assign color spaces to the heads before enabling the output
• implement modules (static, colord) to load color spaces
• compositor plumbing to enable/disable color management
• basic ICC vcgt support
• implement the protocol and extend the surface commit state
• lower color space conversions to a 3D LUT using lcms2
• extract linearization LUT from output color spaces
• gl-renderer changes
• gl-renderer tests to make sure we don't break everything
• !344 (merged)
• !287 (merged)
• actually test different buffer types, formats etc
• 3D LUT support for color space conversions
• gamma correct blending
• fp shadow buffer and/or GL_EXT_shader_framebuffer_fetch_non_coherent
• only assign surfaces to planes when the color space matches the output
• shared CRTC clone mode only for heads with the same color space

It should be implemented because

(see above)

What are the alternatives?

No response

Addtional Information

for more context of past discussion within the x11 project -
https://gitlab.freedesktop.org/xorg/xserver/-/issues/1037

Extra fields

• I have checked the existing issues
• I have read the Contributing Guidelines
• I'd like to work on this issue

[mSparks43]

https://yedlin.net/DebunkingHDR/index.html

further, we already have colord

[sparklescrewdriver]

The video you linked gives a good overview of colour theory. There is the issue of still not having HDR as shown by video content in HDR and SDR. The following videos provide somewhat of an explanation in difference and due to the same video you can see the difference on Wayland between HDR and SDR (if you have a display that supports HDR):

• HDR video

• SDR video

For this project to a future there needs to be a full implementation of HDR (eventually). Some characteristics of HDR is addressed by colord, but depending on how HDR is implemented color management may need to be revamped. Within television HDR is currently set out in ITU-R BT.2100-3. Further, HDR is better set out in some of the standards i.e. HDR10 & HDR10+. Here are some more related links to HDR implementations and questions about HDR:

• HDR 10+ White Paper

• Frequently Asked Questions on
  High Dynamic Range and Hybrid Log-Gamma by the BBC

There has been issues in how HDR has been marketed, but it's a technology that needs to be implemented. As without being able to with confidence say that this X11 implementation supports HDR. It will result in people going to the alternative which is Wayland.

[mSparks43]

see the difference on Wayland between HDR and SDR

The point of the video is you shouldn't be able to see any difference. waylands hdr implementation is broken.

[sparklescrewdriver]

see the difference on Wayland between HDR and SDR

The point of the video is you shouldn't be able to see any difference. waylands hdr implementation is broken.

Wayland's implementation of HDR isn't broken. Rather, it's a choice by the content creator. Quite frankly, we don't have a say in whether we want to support it there is content made in HDR that we should support. Furthermore, the video you linked was mainly referring to filmmaking, but the HDR display type is more widely used than just for filmmaking.

[metux]

https://yedlin.net/DebunkingHDR/index.html

In case anybody else wondering about the empty page, too: you have toclick into on the big rect to start the presentation video :)

[mSparks43]

see the difference on Wayland between HDR and SDR

The point of the video is you shouldn't be able to see any difference. waylands hdr implementation is broken.

Wayland's implementation of HDR isn't broken. Rather, it's a choice by the content creator. Quite frankly, we don't have a say in whether we want to support it there is content made in HDR that we should support. Furthermore, the video you linked was mainly referring to filmmaking, but the HDR display type is more widely used than just for filmmaking.

Its broken because the colour and brightness should not change, if the color was red in sdr and it shows green in hdr its just broken.

All you should get with hdr is a slightly clearer image in the very dark areas that sdr just maps to black.

[sparklescrewdriver]

Its broken because the colour and brightness should not change, if the color was red in sdr and it shows green in hdr its just broken.

The key point is that while a color's tone can change, it remains fundamentally the same color. Ultimately, content creators working in HDR determine how their material should be viewed. Additionally, SDR content maintains its consistency since HDR processing typically isn't applied to SDR material when properly implemented.

All you should get with hdr is a slightly clearer image in the very dark areas that sdr just maps to black.

That was the intent, but like with any technology it can be used in other ways. Thus, if it doesn't have negative effects - it should be implemented.

[Quackdoc]

"HDR" as a focus point is IMO wrong, HDR support will come with proper colormanagement pipelines, all "HDR" really is, is a marketing term for colorspaces which utilize transfers that can represent 1000nits+

the basic "minimum checklist" for HDR is

• Gamut mapping (gamut expansion in 99% of cases)
• (inverse) Tonemapping SDR -> HDR
• Getting metadata from applications (having vulkan wsi support is necessary for good support for games and such, could be a layer but hopefully mesa wont be actively hostile to XLibre)
• being able to set the transfer/colorspace being sent to the display

Things like HDR10/HDR10+/Dolby vision can IMO come afterwards.

The common transfers in question are PQ and HLG. scRGB is common in games, Displays typically don't support this it gets mapped to PQ or HLG. HLG is probably the easiest to start with, but has the worst display support, so PQ is probably the better one to start with.

[mSparks43]

One thing I would suggest is that an X11 extension to add vulkan layers to windows would:

1. have a lot of other use cases
2. Allow native functionality similar to valve coded up with gamescope.
3. be a good use case for xnamespace

[metux]

One thing I would suggest is that an X11 extension to add vulkan layers to windows would:

Any suggestion how it would look like ?

Actually, I wonder what's still left to do here at all:
Vulkan is just an frontend API (state tracker) - at least in Mesa (no idea what Nvidia's doing here), it all goes into the same pipeline. The interaction with the windowing system is just about creating/passing/syncing buffers - that's what DRI is for.
Haven't checked all the details yet, but to me it all seems to be on Mesa side.

[mSparks43]

One thing I would suggest is that an X11 extension to add vulkan layers to windows would:

Any suggestion how it would look like ?

Actually, I wonder what's still left to do here at all: Vulkan is just an frontend API (state tracker) - at least in Mesa (no idea what Nvidia's doing here), it all goes into the same pipeline. The interaction with the windowing system is just about creating/passing/syncing buffers - that's what DRI is for. Haven't checked all the details yet, but to me it all seems to be on Mesa side.

vulkan has the option to add vulkan layers to the pipeline, these can do things like debugging, compute, change colorspace turn output into a video stream whole load of stuff I've only barely looked at.

I do know this is how gamescope is adding its hdr layer to wine X11 games

Its also how reshade works on windows.

I imagine an X11 extension would let an application check for permission then do all this on the x server it is connected to.

[Quackdoc]

It's not hard to do vulkan layers, but first xlibre needs to support it getting colorspace data from the application, triggering HDR on the monitor, and doing the basic transforms.

Context because the terms HDR and SDR are dumb.

a color space is Gamut/primaries + transfer/gamma + whitepoint. As far as I, and the compositor cares, the most "unique" part here is the transfer and gamut. the "HDRness" of colorspaces as far as I am concerned, and most users are concerned, comes from the transfer, but this isn't really relevant because content is done in colorspaces

Most legacy un-colormanaged applications are either bt.1886(rec709) which is common on TVs, or sRGB which is more common on PC displays. These are "SDR" colorspaces. There are others, but for uncolormanaged or "SDR" applications, this is all we really care about. Colormanaged applications can be treated differently (and should be.)

The common HDR colorspaces are rec.2100 with either HLG or PQ transfers. HDR metadata can be added onto it to modify them, but it's not a hard requirement to handle. when it comes to displaying, HLG is probably the "easiest" to do, but PQ is by far the most common.

Games often use scRGB colorspace which is a linear transfer with the primaries of sRGB and doing some weird hijinks with values to expand it. You need to convert scRGB to rec2100 with either of the transfers because no monitors use it. (linear is a lot of data :D)

the current method compositors seem to be going with now to implement SDR->HDR is to have a dedicated SDR plane on the GPU, then use operations to transform that into "HDR" Presumably using a lut, but I don't know what all operations are available to work with here (https://melissawen.github.io/blog/2025/05/19/drm-info-with-kms-color-api).

The actual specifics of the mapping should be user configurable (or compositor configurable, not really sure here) There are a lot of more nuanced things to think about like 'to what luminance do you make "SDR" to "HDR" ' which really should be user configurable.

I think modeling the color data based on the wayland protocols themselves is fine, they are sufficiently detailed and staying in line with them will only serve to help applications port to xlibre.

[mSparks43]

generally agree, but a lot of that is also already done a very long time ago, but the documentation is poor or non existent or just missing. a good place to start is probably the colord source code - what is in there already, what can be hardware accelerated with e.g. vulkan layers, what needs improving/adding.

Oh, and who is going to pay for the $20,000 displays you need to actually barely see if it actually works..

color is a very, very hard topic, these "hdr on" toggles wayland compositors are adding are 100% gimmic, 0% useful. You can tell that by the only calibration software for Linux is X11 only.

[rc-05]

Could be the DeepColor X11 extension proposed by NVIDIA be a potential starting point for implementing HDR support?

[metux]

Could be the DeepColor X11 extension proposed by NVIDIA be a potential starting point for implementing HDR support?

Perhaps. But maybe we don't even need to go that far, if we can live with 10bpc and 2 bits alpha.
The major problem here is current protocol isn't designed color values above 32bit . But 10bpc would still fit in.

Not sure what exactly they're wanted to transmit via window properties. If it's about frame/scene specific gamut, then properties might not be the ideal way to do that. Needs some deeper thoughs.

[metux]

It's not hard to do vulkan layers, but first xlibre needs to support it getting colorspace data from the application, triggering HDR on the monitor, and doing the basic transforms.

you seem to be the right one for writing an specification draft :)

[rc-05]

Perhaps. But maybe we don't even need to go that far, if we can live with 10bpc and 2 bits alpha. The major problem here is current protocol isn't designed color values above 32bit . But 10bpc would still fit in.

If I am not misremembering (please correct me if I am wrong) the Deepcolor extension proposes a VisualClass specifically designed for non-SDR windows.

Not sure what exactly they're wanted to transmit via window properties. If it's about frame/scene specific gamut, then properties might not be the ideal way to do that. Needs some deeper thoughs.

Perhaps over NVIDIA someone thought that the most straight forward way of querying a X11 client's capabilities of offering HDR content and such was through X11 Atoms? 🤔

[metux]

If I am not misremembering (please correct me if I am wrong) the Deepcolor extension proposes a VisualClass specifically designed for non-SDR windows.

To be very precise: it's actually just about increasing the color resolution: how many different brightness levels you have per channel. It doesn't give you a better black or white - only the physical display can do that (by the way, professional cinematic content usually doesn't even get near to the edges of SDR space).

The actual magic of HDR is adapting coloring on the fly, per scene or frame. We just now have an encoding format that needs those things (SDR doesn't), because it's based on absolute instead of relative brightness. Meanwhile they've learned that it practically doesn't work so well, eg. because our visual perception is relative instead of absolute (indeed, many important biological mechanisms are edge-triggered, not level-triggered), so they added new post-filters (eg. in the displays) for trying to adapt to ambient conditions, so make it relative again - and damaging the artistic intent on the way.

Right now, I'm not convinced that's really needed at all.
But increasing color resolution to 10 bits seems a good step forward.

Perhaps over NVIDIA someone thought that the most straight forward way of querying a X11 client's capabilities of offering HDR content and such was through X11 Atoms? 🤔

Not atoms, properties. (atoms are just assigning numbers to fixed strings - a simple way of dictionary compression).
This would make sense if one's not going to introduce additional requests at all. But the drawbacks are eg.

• mix up property namespaces of user/client defined by server-defined things (now suddenly lots of names are becoming reserved) --> can hurt backwards compat
• now any 3rdparty who might be interested in updates (eg. external compositor) now needs to listen on all properties
• sending metadata per frame is complicated

[rc-05]

But increasing color resolution to 10 bit seems a good step forward.

Xorg for instance already supports 10 bit color depth but sadly lots of applications assume instead that the default bit-color depth is 8 bit. Chromium-based programs and Firefox suffer the most from this assumption.

Not atoms, properties. (atoms are just assigning numbers to fixed strings - a simple way of dictionary compression). This would make sense if one's not going to introduce additional requests at all. But the drawbacks are eg.

• mix up property namespaces of user/client defined by server-defined things (now suddenly lots of names are becoming reserved) --> can hurt backwards compat
• now any 3rdparty who might be interested in updates (eg. external compositor) now needs to listen on all properties
• sending metadata per frame is complicated

I did not consider that using properties could be a liability to backwards compatibility... So the better way would be to instead just design and propose an ad-hoc request for this?

[metux]

But increasing color resolution to 10 bit seems a good step forward.

Xorg for instance already supports 10 bit color depth but sadly lots of applications assume instead that the default bit-color depth is 8 bit. Chromium-based programs and Firefox suffer the most from this assumption.

Do you happen to have some example code for testing ?

I did not consider that using properties could be a liability to backwards compatibility...

Well, it could - if somebody's using the same names for something else. No idea whether it actually happens practically.
So at least, this needs to be very well thought. Perhaps we should only do that once by reserving some name prefixes for extension use (and then allocate sub-prefixes for individual ones).

So the better way would be to instead just design and propose an ad-hoc request for this?

If we already need to create a protocol extension and adding new opcodes, then I'd go that way.
It's consuming one major opcode anyways.

I've been thinking about using rootwin properties for Xnamespace extension, but still need to think really careful about that.
(for now it doesn't have dynamic configuration yet).

[rc-05]

Do you happen to have some example code for testing ?

No, sadly. However, it is possible to see how Chromium-based programs and Firefox suffer from using 10-bit color depth.
Firefox will slow down to a crawl (especially noticeable with higher refresh displays) and Chromium will display the wrong colors. Steam will outright crash if 10 bit color depth is enabled, probably because of the afromentioned assumption about the color depth being set to 8 bit.

See the Arch Wiki page for the AMDGPU driver. I do not know if this problem is present with NVIDIA GPUs.

Looking back, this is not really a X11 problem per se but rather a client one due to the 8 bit color depth assumption of said X11 clients.

[Lyky35]

While i may not be correct on all points, what you want is to detect if application wants to send HDR metadata, understand the metadata signal, and if you are in HDR environment that's great, don't force it to applications to be in 10bit, and vice versa for SDR.

• But my colors, my brightness etc

Well if you are in SDR and you have 10bit content use tone mapping to make them look good on 8bit.
If your are in HDR and software doesn't do 10bit, it will still look fine. But allow user to create his own tone mapping / color profile for application or globally. i.e. adjust contrast, and brightness. If tone mapping goes anywhere then same thing can be used for SDR content to up it to HDR per your monitor profile.

(Browsers chromium based, firefox do not really support hdr and they do not send hdr metadata - their implementation is broken because they don't send metadata, and they do try to send 10bit color space instead.)

The wrong colors *"completely bugged colors" appear because they are trying to use wide gamut, and x client, windows, wayland whatever doesn't support wide gamut they understand only sRGB range. (mac does use wide gamut.) Thus its likely overflowing the bit section allowed for color information of pixel. *The correct solution would be to allow user to select color profile (sRGB, Adobe RGB / wide gamut, rec2020 and so on) they are working with; and clip color data on the range their profile supports with nearest maximum in your color gamut.

*if this is done right, and you can select correct color gamut - xlibre will become the choice for photographers hehe.
(I think support for wide gamut is the golden standard, and the way HDR should really be approached.)

[Quackdoc]

I think you are mixing up some terms.

bit depth is not directly related to color. a 0.5 rgb in 8 bit and in 10 bit are the exact same color. "HDR" as the term is commonly used is not strictly contingent on having 10bit colordepth or greater.

When it comes to display output, you have 1 colorspace. You cannot just mix images and expect them to look good. if you have a wide gamut display pipeline, you need to actively convert sRGB to the wide gamut.

Unlike the first example an 0.5 rgb is a different color when comparing gamuts. The display server, in this case xlibre needs to manage transforms between gamuts and transfers.

EDIT: and ofc, xlibre would ideally be able to handle mixed clients, IE. clients that do 8bit and 10bit

[rc-05]

But allow user to create his own tone mapping / color profile for application or globally. i.e. adjust contrast, and brightness. If tone mapping goes anywhere then same thing can be used for SDR content to up it to HDR per your monitor profile.

This could be perhaps done together with colord, since that allows a user to choose the color profile?
I am just brainstorming ideas, as always, correct me if I am wrong or there is a better approach. 🙂

I think you are mixing up some terms.

bit depth is not directly related to color. a 0.5 rgb in 8 bit and in 10 bit are the exact same color. "HDR" as the term is commonly used is not strictly contingent on having 10bit colordepth or greater.

When it comes to display output, you have 1 colorspace. You cannot just mix images and expect them to look good. if you have a wide gamut display pipeline, you need to actively convert sRGB to the wide gamut.

Unlike the first example an 0.5 rgb is a different color when comparing gamuts. The display server, in this case xlibre needs to manage transforms between gamuts and transfers.

EDIT: and ofc, xlibre would ideally be able to handle mixed clients, IE. clients that do 8bit and 10bit

I see... Thanks for the clarifications! 🙂

[Lyky35]

bit depth is not directly related to color. a 0.5 rgb in 8 bit and in 10 bit are the exact same color. "HDR" as the term is commonly used is not strictly contingent on having 10bit colordepth or greater.

If you truly have 10bit image, your whites, your blacks or any other colors won't match unless you can understand / translate 256 x 256 x 256 color matrix into 1024 x 1024 x 1024 color matrix. The color white in 8bit is represented as 255 x 255 x 255 while same white in 10bit is represented as 1024 x 1024 x 1024. You might be able to get near the right color but you won't get same color.

The HDR as it is functioning right now on windows is 8bit of color data + 2bit in separate metadata that typically describe how bright that color should be with 4 settings (0% 25% 50% 100%) - (i might be wrong, but playing with metadata - i.e. changing those 2 bit values produced exactly that - the color depth - generation of more colors is typically only an illusion by dithering.)
*(i'm running a decent monitor pg32uqxr which has wide gamut mode, and does 10bit natively - so its not just a limit of a certain monitor)

When it comes to display output, you have 1 colorspace. You cannot just mix images and expect them to look good. if you have a wide gamut display pipeline, you need to actively convert sRGB to the wide gamut.

To correctly display, and use 10bit color or more we would have to sit on wider gamut and translate the lower and higher bit address matrix to assign the correct color value from the gamut the display handles. Thus it would be better to work in 32bit address space for color depth even though we won't be using it.

Unlike the first example an 0.5 rgb is a different color when comparing gamuts. The display server, in this case xlibre needs to manage transforms between gamuts and transfers.

Yes.

I am just brainstorming ideas, as always, correct me if I am wrong or there is a better approach. 🙂

Thats right i'm not what you call a professional either. Just enthusiast hoping for something good. - and reason to not buy mac or use windows kvm to just play hdr content.

[Quackdoc]

I think implementing the DeepColor extension is a good idea or at least being based on it. Realistically speaking, I think supporting FP16 is desirable anyways and since xlibre will need to put in some effort in supporting mixed clients anyways, it seems like a good idea to do that while keeping FP16 RGBA formats in mind.

[callmetango]

Thank you very much for your contributions! We currently restructured the "Ideas" discussions and accordingly this discussion will be moved to the Good Ideas For Later category.

As far as the matter itself is concerned, we will defentively have a look at it.