Skip to content

Commit f618603

Browse files
committed
[MSE][GStreamer] Initial support for multiple video tracks on MSE
https://bugs.webkit.org/show_bug.cgi?id=298245 Reviewed by Philippe Normand. Multiple video tracks in MSE are (optionally) supported by the spec[1], but have never been used in practice in WebKitGTK/WPE. Creating more than one video SourceBuffer causes a not-linked pipeline error in WebKitMediaSrc. It would be nice to have a basic support of that use case, since it's apparently required for the YouTube EME tests. See: #1549 This patch sanitizes the video track selection state on creation (only one video track can be active at once, all of them appeared as active on creation before), mirrors the selection state from the video tracks known to the HTMLMediaElement to the ones knows by the player private (they're a different set in the case of MSE), and adds support for unregistered streams in WebKitMediaSrc (the ones causing the not-linked error in the first place), just avoiding pushing buffers to pads with unlinked steams. This patch only provides an initial support. It doesn't address complex scenarios such as the init segment and data reenqueues that should be needed if the switch happens after the init segment and data have been appended (and enqueued) and the track selection change happens after that. In such cases, the pipeline is refactored but, since the needed data hasn't reached the video decoder on time (because the decoder just didn't exist at enqueue time), only future samples will be enqueued from that time on and decoding artifacts will happen. A layout test has been added to test multiple video track basic support, ensuring that the track ids are properly generated and that only the first track is selected. [1] https://w3c.github.io/media-source/#dfn-sourcebuffer-configuration Layout test based on code provided by: Filipe Norte <[email protected]> * LayoutTests/media/media-source/content/video320x180-101-1.m4s: Added. * LayoutTests/media/media-source/content/video320x180-101-2.m4s: Added. * LayoutTests/media/media-source/content/video320x180-101-3.m4s: Added. * LayoutTests/media/media-source/content/video320x180-101-init.mp4: Added. * LayoutTests/media/media-source/content/video320x180-102-1.m4s: Added. * LayoutTests/media/media-source/content/video320x180-102-2.m4s: Added. * LayoutTests/media/media-source/content/video320x180-102-3.m4s: Added. * LayoutTests/media/media-source/content/video320x180-102-init.mp4: Added. * LayoutTests/media/media-source/content/video320x180-103-1.m4s: Added. * LayoutTests/media/media-source/content/video320x180-103-2.m4s: Added. * LayoutTests/media/media-source/content/video320x180-103-3.m4s: Added. * LayoutTests/media/media-source/content/video320x180-103-init.mp4: Added. * LayoutTests/media/media-source/media-source-multiple-video-tracks-expected.txt: Added. * LayoutTests/media/media-source/media-source-multiple-video-tracks.html: Test that created 3 different SourceBuffers, loads videos with 3 different track ids, and checks that the proper ids are generated and that only the first video trac> * Source/WebCore/Modules/mediasource/MediaSource.cpp: (WebCore::MediaSource::addVideoTrackToElement): Select the first video track added and unselect the next ones to ensure that only one video track is active at a given time after adding several ones. * Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp: (WebCore::VideoTrackPrivateGStreamer::setSelected): Propagate the "selected" state to the mirror track that the player private may have. * Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp: (WebCore::MediaPlayerPrivateGStreamerMSE::mirrorEnabledVideoTrackIfNeeded): Propagate the "selected" state from a given VideoTrackPrivateGStreamer (typically one know by HTMLMediaElement) to the internal VideoTrackPrivateGStreamer managed by > * Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Added mirrorEnabledVideoTrackIfNeeded() empty implementation. * Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h: Added mirrorEnabledVideoTrackIfNeeded(). * Source/WebCore/platform/graphics/gstreamer/mse/WebKitMediaSourceGStreamer.cpp: (WebKitMediaSrcPrivate::streamById): Support asking for streams not registered in this element. Log an info instead of asserting when not present. (webKitMediaSrcTearDownStream): Gracefully bail out if the Stream is not registered in this element. (webKitMediaSrcLoop): Ditto. Just skip the buffer push without triggering an error. (webKitMediaSrcFlush): Ditto. Canonical link: https://commits.webkit.org/299700@main
1 parent f5892cd commit f618603

20 files changed

+159
-10
lines changed
36.6 KB
Binary file not shown.
36 KB
Binary file not shown.
35.8 KB
Binary file not shown.
942 Bytes
Binary file not shown.
36.6 KB
Binary file not shown.
36 KB
Binary file not shown.
35.8 KB
Binary file not shown.
942 Bytes
Binary file not shown.
36.6 KB
Binary file not shown.
36 KB
Binary file not shown.

0 commit comments

Comments
 (0)