Is it possible to append a frame to an existing video?

[burkaman]

I'd like to append a new frame to the end of an existing video file. I'm not sure if this is possible (but I haven't figured out how), not yet possible (but worth me working on a PR to support it), or impossible (not really how ffmpeg works, or would require a major rewrite of this package or something).

I first tried simply opening an existing video with imageWriter and writing a single frame, which overwrites the whole video with just the new frame. I'm assuming this is the expected behavior.

Next, I tried reading all the frames of the existing video with imageReader, and then writing them all back out, plus the new frame, with imageWriter. This almost works, but with two significant problems. First, it's obviously very slow, but also, my method of grabbing all the frames seems to drop 2 of them? Given a 125 frame video, it reads 123 frames, and then adds my new frame, so every time I run the function my video actually ends up one frame shorter. I haven't yet determined which frames I'm failing to read.

Here's my code (somewhat simplified so I'm not 100% sure it compiles, but hopefully it gets the point across):

addFrame :: Image a -> IO ()
addFrame image = do
  initFFmpeg  
  (reader, cleanup) <- imageReader (File "/path/to/video.mp4") :: IO (IO (Maybe (Image PixelRGB8)), IO ())
  oldFrames <- unfoldM reader
  cleanup
  let newFrames = oldFrames ++ [image]
  writer <- imageWriter (defaultParams width height) "/path/to/video.mp4" :: IO (Maybe (Image PixelRGB8) -> IO ())
  mapM writer (map Just newFrames)
  writer Nothing
  return ()

So I have two questions:

1. Why does unfoldM reader miss two frames of the video? If I run this function repeatedly, video.mp4 gets one frame shorter every time. And if I check the number of frames with ffmpeg before running the function and then print length oldFrames, it's always 2 frames shorter than what ffmpeg reported.
2. Is there a way to avoid this and directly append the new frame? If it's possible in theory I'd be willing to work on a PR, but if it's out of the scope of this package then I'll find another way.

[acowley]

The dropped frames sure sounds like a bug! We should pull that out into a separate issue. It's probably to do with time handling as this is very difficult.

As for your desire, it's worth figuring out if ffmpeg lets you do something like this and have it work quickly from the command line. If so, we could perhaps replicate it. Maybe it's something like making a one-frame video of your new image, and then appending videos using the ffmpeg API.

Given what exists today, the approach I would have taken is to do as you did and read in all the frames from the original video, write them all out, then write out the new frame. That dropped frame thing sounds bad, so we should try to isolate it in case it depends on the video format or something fragile like that.

[burkaman]

Thanks for responding so fast!

Ok, I'll make a separate issue for the dropped frames. Good to know I wasn't missing anything obvious in the API. I'll look into how you do this on the command line; I think it's similar to what you said, you just make the image into a video and then concat the two videos, but there's like 5 different ways of concatenating videos. You can probably close this, and I'll open an issue or a PR if I find anything promising.