Dual-Video: No Audio, no Waveform when Slides have no Audio

[marcusm7]

Steps:

• Create an event and upload a dual stream (slides + cam) video
• The slides have an audio track but the track is mute (this happens when you record events i.e. with the Extron SMP352 - there is always an audio track but not always audible audio)
• The cam has an audio track with audible audio
• Both audio tracks are AAC Stereo
• Create the event with straighttopublish: false
• wait until the ingest and preprocessing is finished
• open the editor for the video and click cutting -> screenshot
• Temporarily deleting the presentation Stream activates the audio but the wavewform stays absent (warning: this makes the error-behaviour go away for the no audio part)

Error:

• Sometimes (I don't see a system, maybe it depends which video is first in the ingest workflow), not always -- the audio from the cam-stream is not audible and the waveform is missing. It looks to me, as if only the mute audio track is played.
• If you don't see the error: Repeat the steps above until you get the error.

Expected:

• Perhaps create a sum of the audio tracks for the editor or add a selector, which allows the user to choose whether to playback one or both audio tracks.
• It would make sense to only playback the audio-tracks of the active video streams. I can hear the cam audio even if I delete this stream. - I didn't think this through til the end. It's just a remark. :)

Screenshot (bad):

Screenshot (good):

More:
Looks a bit like #1380 - But that issue seems to be fixed and merged into the most recent Opencast (17.1)

[marcusm7]

Oh:

Version:

• Most recent Firefox
• Opencast 17.1 on stable.opencast.org

[Arnei]

Thanks to #1376, the editor should be smart enough to play back the audio from a video that actually has audio, based on information it got from Opencast. So if Opencast tells the editor there is an audio stream, but then there actually is not, well, not much the editor can do about it.

I would still expect the editor to render a flatline instead of no waveform at all, but I'm actually not sure how the waveform generation interacts with "mute" audio tracks.

Perhaps create a sum of the audio tracks

Iirc some people use Opencast workflow operations for that. For example, if they know that they always want the presenter audio and never the presentation audio, they apply the presenter audio to the presentation video so both videos have the same audio.

add a selector, which allows the user to choose whether to playback one or both audio tracks.

Could be good, but might not be as straightforward as it first sounds. Will have to

• Figure out some good ui for it
• Does that mean we also need to generate a waveform for each track, per default? Waveform generation is already a pain point due to taking a lot of time and CPU.

At least the code changes should be mostly easy enough.

[Arnei]

For your case specifically, it might be the easiest and fastest to adapt your Opencast workflows instead of waiting for editor changes, if you know beforehand what you'll want to do with your audio tracks. E.g. if you already know that you never need audio track from the slides, use a workflow operation to drop it or override it before going into the editor.

[lkiesow]

Also a -1 to just playing both tracks in parallel. We tried that in the past, but most CAs record audio in both tracks. That makes sense, since e.g. users can download single video streams. However, that can easily lead to an echo effect if you just play both streams.

Identifying the one with audio and playing that therefor makes sense.

Indeed, this is problematic if you have a muted audio track. But if this is always done by your capture agent, I would second Arne and suggest dropping the track in the workflow as one of the first operations. You can also use the prepare-av operation to copy the audio to the second track.