Live Stream Downloads: broken files

[televischn]

Your question

Hi together,

I'd like to record live streams. VDH (Firefox add-on) does find an .m3u8-link. For shorter durations video recording mostly works. In case of longer recordings (several hours) some problems arise:

• with the current VDH-version (that I tried before) it happens that a recording is started and running (whitch can be seen: a blue shaded "1"). To stop this download I cancel the running action. It now happens, that nothing happens: the blue shaded "1" remains and the download is not being finished also after longer waiting. Then, when I once more cancel the download, a file is being created, yet obviously only a recording since the first "cancel"-click (not since the click on "Download").
• this does not happen with an older VDH-addon-version (see version details))_

Issue:

• with both the new and the older version there is another problem: after recording a live stream and cancelling to stop recording, a file is being saved (size several GB), but quite often the file cannot be played (I use VLC). Obviously the recorded video is broken (kind of). This seems more often to be the case with longer recordings/bigger files (e.g. 10 GB), but there are also big files that are not broken and that can be played.
• I've already tried to fix/to repair these files with VLC or some "Digital Video Repair" (but Digital Video Repair seems to be Adware/Spyware!), anyway I wasn't lucky/didn't manage to repair the mp4-video-files.

My questions:

• What is the cause that live stream recordings are broken?
• Is there a possibility to repair these files?

Hoping you can understand and help with that problem.
Thank you very much in advance!

Addon & CoApp version and details

version: 8.1.4.0
beta: false
build date: 2024-01-04
build options: linuxlic: false, noyt: false, browser: firefox
lang: de
license: unset
platform: x86-64 win
UA: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0
coapp: yes. Info:
{
"id": "net.downloadhelper.coapp",
"name": "VdhCoApp",
"version": "2.0.10",
"binary": "C:\Program Files\DownloadHelper CoApp\vdhcoapp.exe",
"displayName": "VdhCoApp",
"description": "Video DownloadHelper companion app",
"target": {
"os": "windows",
"arch": "x86_64",
"node": "18"
},
"home": "C:\Users\Johann",
"converterBinary": "C:\Program Files\DownloadHelper CoApp\ffmpeg.exe",
"converterBase": "ffmpeg",
"converterBaseVersion": "6.0"
}

[Mattmjs]

Where does this happen on what site, this is not something I've seen happen much since I'm not recording a lot of long live streams.
But are you left with an mp4 file that is unplayable or is it a .part , which would be a temporary file.

[televischn]

For example sites you could e.g. try www . dmf . tv
or https:// de-de . wedotv . com/channels?program=filmgold
etc.

What remains after recording a live stream with Video-Download-Helper is an .mp4-file (with e.g. 1 or 10 GB) (not a .part-file). When opening this file with VLC Player nothing happens. When having activated details in the Windows folder like length or frame width the files that are not functioning can be identified, as there are no values in these columns...

[Mattmjs]

I tried the first link and let it download for a short duration then stopped it, which produced a file that plays.
The download speed is heavily throttled, it would take a few hours to try to replicate the issue. What I did though is start another
download and let it go for a couple of minutes. I clicked end task on ffmpeg in the task manager.
This left an mp4 file with no duration and no information in the properties. Checking the file in ffprobe says moov atom not found.
I think this is what happened to your file, and it can't be repaired.

Maybe because the file size is so large, stopping the download will take some time to finalize the file. It's a theory, I don't know
for sure. Try stopping the download once and just leave it to see if it will finish. If it still doesn't do anything, try turning off internet.
This will cancel the download and hopefully produce a usuable file.

[Mattmjs]

More testing in ffmpeg itself using a video on Odysee, I started a download in different formats mp4, ts and mkv.
Then after downloading 15 minutes of the video I improperly terminated ffmpeg from the task manager.

The results :
An unusable mp4 file with a moov atom not found.
The ts file was playable, it had a time and information in the properties.
The mkv file has no time or information in the properties but is still a playable file like the ts file.

VDH has a download and convert to mkv but is a premium feature that requires a license. If you have the license and recording over
a few hours choosing download and convert to mkv would be a safe option.

[televischn]

Thank you Mattmjs for your competent answer and your hints!
I now also tried ffprobe (ffprobe -show_entries format=format_name,format_long_name -of default=nw=1 -v error C:\folder\test.mp4) and it also said "moov atom not found" and "Invalid data found when processing input".
This seems a quite common problem with mp4-files as my search with Google showed. There seems to be no clear solution by now but some approaches. Perhaps I'll try Untrunc, whether this helps repair the corrupted video files.

I also assumed that it may take some time to finish a stopped download, but the broken files seem to occur also when waiting a longer time. (I also have tried different hard disks/SSDs but that didn't solve the problem.)
Perhaps it could be worth a try stopping recording by turning off internet/reconnecting internet. I'll test that next.

How did you record the .ts file, with VDH?

[Mattmjs]

I'm not sure a file can be repaired if the moov atom is not found but you can try and see. How did I record the .ts with VDH.
VDH doesn't have that option, I used ffmpeg itself. The old companion app before version 2 had the option to save file as
M2TS which is similar to a ts file. It's another option if you know how to use ffmpeg. MKV and ts files appear to be safer options
should something go wrong.

[GrampaWildWilly]

I do not work for VDH. I'm just another user.

I have also intermittently had trouble with VDH terminating recordings of livestreams. Other users have also posted here about this issue. It seems to be an intermittent problem. Sometimes, you can click the red Stop button in the VDH interface & the recording terminates normally & the resulting file plays fine. Sometimes, the recording stops & immediately resumes, replacing your multi-gig file with something new, which you cancel & to your dismay, you now have a 5-second clip of what's happening now instead of the 5 hours you had been recording successfully. Sometimes, it simply screws up, as you have experienced.

To be safe, when I am using VDH to record a livestream, I pull the plug on my Internet connection. I physically unplug the electricity feed to my Internet hardware. This makes VDH think the livestream has terminated normally & then it saves the MP4 information -- the MOOV atom -- that makes the file playable. You can verify the playability of the file, then plug your Internet connection back into electricity.

Much more often, I record livestreams using ffmpeg directly. This gives me access to several parameters that VDH does not. This is ironic because under the covers, VDH uses ffmpeg to record livestreams . . . indeed, to download everything, live & otherwise. I use the ffmpeg parameters that control duration of recording & detection of the end of the livestream. I have posted elsewhere about this. Search this forum for Plan B & you will find some examples dealing with livestreams.

I have found that while an MP4 is being recorded -- by VDH, by ffmpeg directly, by any other means -- that you can't play the MP4 while it is being recorded. In contrast, if your target file type is either MP3 (audio only) or MKV, you can play that while it is being recorded. Of course, you can play only so much as has already been recorded, but you can play what you have while the rest of it is being captured. That means that your position in the file during playback is chasing the position where the recording is continuing. That later position is, of course, always moving ahead. The critical difference is the MOOV atom. Media players rely on the MOOV atom when you're dealing with MP4. Such is not the case with the other formats, which is why media players can play those despite their still being recorded.

Neither MP3 nor MKV includes a MOOV atom. That means that failures of such recordings usually leave you with a playable fragment. Since the MOOV atom of MP4 is not written continuously to the file, you are always vulnerable to failures. Cancelling an inflight invocation of ffmpeg to record an MP4 is the equivalent of a failure & leaves you with a file that has no MOOV atom, a type of damage that cannot be repaired, as Matt has already pointed out. If you send the q signal to ffmpeg, or just unplug the Internet as I describe above, the software recognizes this as normal termination of the livestream & it writes the MOOV atom, which leaves you with a playable MP4 file.

[televischn]

Thank you for your answers!
I've tested some more recordings in the meantime, maybe it's a little bit better to disconnect/reconnect internet than to use the cancel-function in VDH. I don't know about another way for a normal ending of the ffmpeg-recording in the background.

I've now also searched for some methods to repair corrupted mp4-video-files I had recorded with Video Download Helper/ffmpeg before.

• I found https://github.com/ponchio/untrunc but wasn't successful installing/using it.
• I also found another explanation (https://www.linkedin.com/pulse/how-fix-moov-atom-found-error-handy-recovery-1bk4f/) and reference to https://github.com/SuslikV/untrunc-w. I tried that and in my test it was able to repair 4.079.105 of a 4.076.289 KB-file but with only 2:05:25 duration and with very asynchronous audio (and parts muted).
• And I found https://github.com/anthwlock/untrunc which besides has a GUI. It worked pretty fast (much faster than untrunc-w). It recovered 4.082.190 KB of a 4.076.289 KB-file with 3:00:40 duration, and with synchronous audio.

The last one (anthwlock untrunc) seems to be a good way to make mp4-files playable again (with error: no 'moov' atom found).

With another file I found that anthwlock-untrunc could also repair much (or all) of the video but audio was little asynchronous/video had some delay.
So far my testings for now. I still want to test other recording formats with VDH or else, if possible.