Add a new blackbar detection mode for subtitles

[Jerroder]

Feature request

What problem does this feature solve?

When watching a movie with blackbars, subtitles un-trigger the detection and top and bottom LEDs are turned of, except for white where the subtitles are.

What does the proposed API look like?

How should this be implemented in your opinion?

Instead of detecting blackbars based on the whole red area, the bottom zone should be split in the middle, something like the green area

Are you willing to work on this yourself?

Sure I can write a little C++, but testing is a bit more complicated since HyperHDR is running on my webOS TV.

[notjosh]

I was looking into this a little for myself, as I was having the same issue. I switched my blackbar detector to use letterbox mode and it solved (most) of my issues. (The only issue that remains is the case represented in your screenshot when there are 2 lines of subtitles - the bottom line is successfully ignored, but the top line will form part of the "image" and cause more white lighting than just the video content alone. This is a reasonable compromise though IMO.)

The original PR (here) that implemented the letterbox mode has a good screenshot to demonstrate how it works - at the top it checks points at 25%, 50%, and 75% across, and at the bottom only 25% and 75% (to avoid subtitles interfering, similar to how your screenshot suggests):

However I propose that the thresholds at the bottom should be user configurable - 25%/75% is a reasonable default, but I'd like to be able to use 15%/85% or 20%/80% for some content. And perhaps the documentation can clarify that it does actually make an effort to avoid the subtitle region.

I almost think there's an argument to rename letterbox mode to something like cinema (i.e. implying it's not just the letterbox black bars, but makes room for subtitles in TV/movies - which I think is a better default for most users), and default could get renamed to be more descriptive (like simple or naive). But that's a whole 'nother discussion out of scope of this issue.

[awawa-dev]

Sorry for the silence on the subject for now but currently I'm working on LUT calibration. I have this feature in mind with the fact that... why should we limit ourselves? let the user activate as many scanlines as he needs from the pool available (every 10%) at the top and bottom. this doesn't burden the processor so much even in the case of RPi. Blackborderdetection would have to be completely rewritten, but this is even better. Since I don't use this module I only have a doubt whether we should assume that the blackborder at the top has exactly the same height as the one at the bottom (+/- 2-3 pixels)? And if many vertical scan-lines (from top and bottom) are enough or oblique are still necessary?

[chcore]

The only issue that remains is the case represented in your screenshot when there are 2 lines of subtitles - the bottom line is successfully ignored, but the top line will form part of the "image" and cause more white lighting than just the video content alone.

I know it's not what's being explicitly discussed in this issue, but I would love a solution to this, as I watch most things with subtitles enabled.

A naive implementation might be for the user to specify the color of the subtitle and its outline, and have HyperHDR ignore any pixels near the bottom of the screen that match those two colors when calculating the color to send to the LED. Maybe only do it when pixels matching those colors are grouped together/adjacent and encountered in the same block.

Maybe even instead of ignoring those pixels, it could "borrow" colors from an adjacent block or extend the reach of the block to take pixels from further up in the content.

A local AI based approach would probably fix this problem completely, but I wouldn't presume that is a route awawa-dev would want to go down.

[computergreek]

The only issue that remains is the case represented in your screenshot when there are 2 lines of subtitles - the bottom line is successfully ignored, but the top line will form part of the "image" and cause more white lighting than just the video content alone.

I know it's not what's being explicitly discussed in this issue, but I would love a solution to this, as I watch most things with subtitles enabled.

A naive implementation might be for the user to specify the color of the subtitle and its outline, and have HyperHDR ignore any pixels near the bottom of the screen that match those two colors when calculating the color to send to the LED. Maybe only do it when pixels matching those colors are grouped together/adjacent and encountered in the same block.

Maybe even instead of ignoring those pixels, it could "borrow" colors from an adjacent block or extend the reach of the block to take pixels from further up in the content.

A local AI based approach would probably fix this problem completely, but I wouldn't presume that is a route awawa-dev would want to go down.

this is genius. i basically watch everything with subtitles enabled.

getting hyperHDR to ignore white pixels near the bottom of the screen would be a gamechanger. particularly if it could be toggled on/off in the webapp's remote control view page (like how HDR can be toggled on/off).

this would be a gamechanger for me--and probably a lot of other people who watch their TV and movies with the subtitles on

[Cheerpipe]

Would using Sobel to highlight horizontal edges and then searching for a horizontal vector with a sufficiently high pixel sum be too much computationally expensive?

Example image with sobel detecting only horizontal borders (mainly)

[awawa-dev]

I think the device will handle it but I don't know how it will behave if it comes across a TV logo or movie subtitles: these are not present in this example