[Feature] Support for "Ultra HDR" pictures on Android

[zsolt-donca]

The feature

I noticed when I open my photos on my Pixel 7 phone in Google Photos, the images are displayed in an HDR mode. The way it works is that right after opening the photo it starts out as is "bland", but then just after a fraction of a second it "pops out": the whites become whiter, the colors become more saturated, the whole thing becomes much more pleasing to the eye.

If I check the details of the photo, it states "Ultra HDR":

It would be great if Immich Android app could display these HDR photos in a similar way, with the added brightness and color depth.

Here is a sample photo:

Platform

• Server
• Web
• Mobile

[mertalev]

I'd like to add this, but it would require upstream support. Something more actionable in the short-term would be to support AVIF, since this format allows for 10-bit HDR.

[n00mkrad]

AVIF is irrelevant.

No camera shoots AVIF.

The only relevant HDR formats are JPEG (with UltraHDR gain map) and HEIF (Apple) as well as potentially some raw formats.

[mertalev]

The previews and thumbnails we generate can be AVIF. The fact that the format supports 10-bit HDR with much better compression efficiency than JPEG makes it a good candidate for this.

Of course, the other part of this feature request is to have the mobile app actually display the asset in HDR. Flutter brings its own set of challenges there.

[n00mkrad]

Is there a spec for AVIF with a gain map?

No need to reinvent the wheel in case it does not.

[mertalev]

AVIF officially supports HDR both natively and through a gain map. I think the latter is much less common, though.

[martabal]

Seems like ImageMagick supports ultrahdr https://github.com/ImageMagick/ImageMagick/releases/tag/7.1.1-30 thanks to libultrahdr

[n00mkrad]

Neat but I'm pretty sure immich uses libvips and not IM

[martabal]

Libvips uses imagemagick if it does not support the image format. This is why Immich can decode raw images.

[apparle]

Do we need libvips support just to display the image or only when editing/modifying/resizing it?
If the image is local on my phone and can be displayed in full quality, is there a stop-gap implementation to just display it correctly with proper Ultra HDR brightness by enabling the HDR mode as in https://developer.android.com/media/grow/ultra-hdr/display ?
All new Android phones are adopting this format, and the pictures look bland when viewed through immich app -- the difference is big enough to be a deal-breaker for layman "why do my photos look worse in this app, I don't like it", and it looks like libvips is going to take its time :-(

[mertalev]

libvips is only to generate Ultra HDR preview images. The actual displaying of Ultra HDR in the app is a separate issue that's limited by Flutter. We're switching to a native video player for HDR support to get around Flutter limitations, and might have to do something similar for images.

[apparle]

Is there a separate issue to track the displaying of Ultra HDR, which I should follow?
I'm realizing that I care more about being able to see UltraHDR in the image viewer once I open a stock camera photo, because that's the most common usage.
Thumbnails and editing is all gravy, we should eventually get there -- but I don't need / won't notice on a daily basis.

[bigjam3]

This is a huge deal for me personally, I love my HDR images.

However I’m on iOS, is there a feature request anywhere for iOS and HEIF gain map HDR photos?

[TheBeaconCrafter]

Is Ultra HDR on the roadmap? I'd also love to have this feature available as it's a big deal for me.

[mertalev]

It is planned, but not a high priority at the moment. Displaying Ultra HDR images likely requires us to write a custom image library backed by native Kotlin/Swift viewers, which may or may not also need us to handle gestures, zooming, swiping between assets, etc. We wouldn't need to if Flutter's rendering engine had better HDR support, but they unfortunately haven't prioritized HDR as much as I'd like.

[Delivator]

What about support for the web app? The "full size thumbnail" (not sure what you'd call them), lack ultra HDR support and only show correctly when I start zooming in a little. I assume it loads the original full image when I start zooming in but it would be nice if it would show correctly when I'm viewing photos in fullscreen without having to zoom in for every image.

[M1cha]

Wouldn't it be easier to add HDR support to flutter and create a PR instead of rewriting everything in Kotlin and Swift?

[dunestorm333]

Just purchased the $100 supporter package to try and bump this on the priority list. HDR photos is the one feature holding me back from using Immich as my default image viewer!

[mertalev]

Wouldn't it be easier to add HDR support to flutter and create a PR instead of rewriting everything in Kotlin and Swift?

No, a PR for that would not fit in a single PR, it would be a milestone that involves a lot of planning and touches many things from their rendering engine to their APIs. It's much more feasible to use the existing native support than for us to contribute this functionality to Flutter.

We've been getting more familiar with Swift/Kotlin development and have a more established process for it now, so it's actually not that crazy to use native viewers. A PoC would be interesting to see how well it would work. Still no promises on when something like this would land, though.