Improve Timeline Scrubbing Visibility At Hour Breaks

[nulledy]

Describe what you are trying to accomplish and why in non technical terms
One of the first options I set was for my preview quality to be very_high as I generally scrub the timeline of my important cameras daily. One thing that has always bugged me is that at the hour breaks when a new preview segment mp4 is loaded, the first several preview frames of the new hour are a garbled mess. I'd like to improve that.

Describe the solution you'd like
I have a solution I've been testing, which is to set the -qmax ffmpeg parameter when generating the preview segments.

-qmax sets a quantization ceiling, preventing the encoder from aggressively and excessively compressing frames, which is especially important for the preview I-Frame at the start of the segment. The encoder will still try to hit the target bitrate, but with -qmax set will refuse to compress frames beyond the quantization ceiling, which is especially impactful for the starting I-Frame. This does have the downside of temporarily exceeding the target bitrate, due to the fact that the initial I-Frame cannot "fit" in the target bitrate without excessive quantization (compression).

Describe alternatives you've considered
I also did some testing with the -maxrate, -bufsize and -rc_init_occupancy parameters to try and give the encoder a bitrate buffer budget, however since all of the webp frames we are encoding are significantly larger than the target bitrate this had the unfortunate side effect of the buffer being filled by the early frames, quickly exceeding the target bitrate, and then using excessive compression to get the average bitrate back down to the target bitrate. Moving the problem from the start of the preview segment, to later once the buffer budget was full.

Additional context
If this is something that would be considered, I'd be happy to do some further testing at each of the preview quality presets to try and come up with an appropriate -qmax setting for each preset and then put together a PR. This would be a small change, with a large benefit imo.

Currently I have my qmax set at 20, quality profile set to very_high and have seamless scrubbing between hours. Comparing clips at the same time of day from the same cameras before and after my change I'm seeing a 100KB-200KB increase in hourly preview segment file sizes (~4.5MB-4.6MB vs ~4.4MB during the day) for most segments, but many are actually smaller, likely just due to less activity and therefore less frames.

I think this could be tweaked to get us closer to the target bitrate while still making the initial frames "acceptable".

My expectation would be that other quality profiles would need a higher qmax value to keep us closer to their target bitrates. However, I'm not sure if that would definitely be the case due to the webp frames being lower quality to start with. Additional testing would be needed, which I'm happy to do.

[NickM-27]

I don't have any inclinations against this idea other than it must be heavily vetted and validated. Things we would need to confirm:

1. Mobile (specifically mobile!) scrubbing fluency is not affected. It must still be smooth at every point like it is now. This includes at every quality level
2. Overall quality can not be measurably worse at any point in the preview.
3. There must be no compatibility, performance (in the creation process), or other reduction.

Some ideas to consider:

• Perhaps only apply this optimization for the non-default higher quality levels
• would be great if there was some way to visualize the bitrate throughout the preview and how it is affected by this arg.

[nulledy]

1. Mobile (specifically mobile!) scrubbing fluency is not affected. It must still be smooth at every point like it is now. This includes at every quality level

Completely agreed

2. Overall quality can not be measurably worse at any point in the preview.

Agreed, that's exactly why I moved to qmax vs the bitrate buffer approach which could impact quality once the buffer was filled, while setting qmax lower than the default can only increase quality.

3. There must be no compatibility, performance (in the creation process), or other reduction.> > Some ideas to consider:

• Perhaps only apply this optimization for the non-default higher quality levels
• would be great if there was some way to visualize the bitrate throughout the preview and how it is affected by this arg.

I really like the idea of only doing it at the higher quality levels, especially as a first pass at the feature. Cuts down on initial testing for me as well. I will also see if I can get bitrate statistics, along with corresponding sample preview files.

Thanks Nick, I'll get to work when I have the time and eventually open a PR.

[nulledy]

I'll catalog my testing in this issue to keep the PR about the code changes.

Testing Methodology

The goal is to make a consistent and reproducible test.

• Acquired an hour long sample MP4 video from a publicly available security camera that had a realistic mix of activity and no motion
• Set this video to loop as a test camera in my test Frigate instance
• Programmatically disabled and enabled the camera in Frigate exactly 5 minutes before the start of the hour to ensure the sample video was as close as possible to being at the same place at the start of the hour when the test would begin
• Check the quality of the resulting preview file
• Plot the bitrate of the file using plotbitrate

Test - Preview Quality very_high

qmax = default (none)

Preview File Size: 4,466,425 bytes

Preview File:

no-qmax.mp4

Preview File Bitrate Over Time:

---

qmax = 25

Preview File Size: 4,477,365 bytes

Preview File:

25-qmax.mp4

Preview File Bitrate Over Time:

Conclusion

We can see that the quality at the start of the preview segment is MUCH improved and the size of the file increased by only 0.24%. On the bitrate plot we can see our peak and average bitrates stayed the same and the main noticeable difference is a higher bitrate on the initial frames, which is exactly our goal.

For the very_high preview quality preset, -qmax 25 seems to be the sweet spot. I also tested with -qmax 30 but I did not find the quality at the start of the clip to be ideal and discarded out the results.

[NickM-27]

Thanks for testing, looks good to me

[nulledy]

Test - Preview Quality high

qmax = default (none)

Preview File Size: 4,024,911 bytes

Preview File:

no-qmax.mp4

Preview File Bitrate Over Time:

---

qmax = 25

Preview File Size: 4,025,569 bytes

Preview File:

25-qmax.mp4

Preview File Bitrate Over Time:

Conclusion

Somewhat surprisingly with a quality profile of high we see an even smaller relative increase in file size of only 0.016%, implying that due to the lower quality webp frames we start with there are fewer frames where the encoder would have to use a higher quantizer value than 25 to hit the target bitrate, than with profile very_high. Again, our peak and average bitrates stay the same between the files. Based on this testing -qmax 25 seems appropriate for the high quality profile as well.

Based on the earlier discussion of only initially applying this change to the high and very_high profiles, I don't plan on further testing. However, for future reference it seems plausible that a qmax value of 25 may in fact perform quite well for all of the quality profiles.

@NickM-27 should this be targeted to dev or 0.18? I see it as more of an improvement to an existing feature than a new one but I'm happy to submit the change to whichever branch it is welcome.

[NickM-27]

dev is in RC so there will only be critical bug fixes accepted, this should go to 0.18