MP4 format captures with glitches

[antlii]

Hi!

I found problems with captured MP4 videos, the image starts to glitch when working with the video file in programs like Adobe After Effect / Photoshop / DaVinci Resolve. Just in case I've tried several computers and various versions of editing apps to avoid the possibility of a problem with a particular hardware or software.

As an example, I recorded a one-minute video of the screen, about how the glitches occur using one of the programs. You can see that when you just play the captured video no glitches are detected, but when you work with it problems begin to emerge:

p5capture-mp4-glitch.mp4

It seems to be connected with the h264-mp4-encoder library itself because when I checked the captured video from the demo I found the same problem.

I should probably report the problem to the developer of this MP4 encoder, but I am primarily interested in the functionality of your library, maybe you can find some other solution for making capture works properly for MP4 format.

Thanks!

[tapioca24]

Hi @antlii, thanks for reporting this!

I watched the video and confirmed the problem. I am not familiar with mp4 encoding, so I have no idea what causes this. I think it is most likely a problem with h264-mp4-encoder, so you might want to report the problem there if possible.

Another solution is to export as sequentially numbered images (e.g. png) with p5.capture and then generate a mp4 video offline using another encoder such as ffmpeg. This is tedious because of the additional steps, but may solve the problem.

[antlii]

Yeah, I suddenly found this problem myself, just while working with one of the videos I recorded with the p5.capture. I thought it was something wrong with the settings I used, but then I started figuring it out and realized that it was a problem with the h264-mp4-encoder itself.

I know it's possible to capture in WebM format or as a series of static images and then convert it, but it would still be nice to be able to capture it into the most popular video format directly. So I just wanted to let you know about the problem, that it might be worth using a different encoder for your library if you're willing to do that of course.

Thanks for your work!
With warm wishes from Ukraine 😊

[tapioca24]

I see.
I'm sorry, but the choice of mp4 encoders available in browsers is limited, so using different encoders will be difficult for now. In the future, when the WebCodecs API is officially supported by many modern browsers, this may become a viable option.

Anyway, thanks for reporting this!
I wish you peace in Ukraine 💚

[antlii]

Thank you!
I understand, also thought so, there are not many options for now. I didn't know about WebCodecs API, looks very promising and I hope it will be available soon ;)