Players: Event model for Playback Sync

[iparamonau]

Here's a quote of what we need community help with here:

• playback - for player indicating that widget supports JavaScript or window.postMessage API to control media playback. With the list of calls, similar to what YouTube, Vimeo and Soundcloud have.

So in a sense, the task is to synchronize the spec of those three providers that all of them have probability to potentially adapt in the future. Cross-provider playback sync is critical for a lot of apps. At the moment they have to rely on removing/adding players in DOM to control user experience.

We already have some of the general events in the spec (like the ones that register the iframe and the one that resizes it on onload) - but it can be changed to accommodate the general approach.

[bdougherty]

It seems like all three APIs are pretty similar. I'd propose something similar to what we're all doing now.

---

Events

Event messages are posted from the child window to the parent window.

Event names

The following subset of HTML5 media element events should be used since they cover both audio and video playback:

• play
• pause
• ended
• timeupdate
• progress
• seeked
• error

Message format

Data is an object that contains specific information about the event.

{
    event: 'name',
    data: {}
}

Progress-based events should return the following keys in the data object. Numbers should be specified to 3 decimal places.

{
    seconds: 1,
    duration: 10,
    percent: 0.1
}

Methods

Method calls are made from the parent to the child, with an optional response from the child back to the parent.

Method names

addEventListener(string:eventName)

Messages for a specific event should not be sent from the child to the parent unless this method is invoked.

removeEventListener(string:eventName)

Once this method is invoked, the specified event should no longer trigger messages to be sent.

The following methods are defined according to the equivalent attributes of an HTMLMediaElement:

• play
• pause
• getCurrentTime
• setCurrentTime(double:time)
• getDuration
• getEnded
• getLoop
• setLoop(boolean:loop)
• getPaused
• getVolume
• setVolume(double:volume)

These methods should be supported for video content:

• getVideoWidth
• getVideoHeight

Message format

The following message format should be used for both the message from the parent and the return message from the child.

{
    method: 'name',
    value: 'optional value'
}

---

I've chosen the names for everything based on <audio> and <video> because hopefully that's something everyone can agree on.

After reading through itteco/iframely#29, my current thinking is that embeds should support receiving the messages as both objects and strings, and should respond accordingly. That way browsers that do support sending objects (which is most of them at this point I think) don't have to have the (admittedly small) overhead of JSON.stringify and JSON.decode.

It also is probably worth it to specify some sort of initial communication between the parent and the child. We're still trying to figure out how to fully solve this with our Vimeo embeds, because certain privacy settings (like password protection) result in a player that loads but cannot do anything until the viewer supplies the password.

There are also cases where the player will never load. I think those can be handled with an error event though.

[iparamonau]

Thanks Brad. Appreciate the input - it's a great start!

I have couple questions regarding specific playback events, and some general observations.

Playback:

• Do you think it makes sense to communicate playback quality sets, gets and changes? As user, I am annoyed to turn HD on when the video starts. I'd prefer my platform to remember my choice and deal with embeds' default resolutions for me. The same is for volume, but you have it covered already.
• I know Vimeo doesn't, but if publisher gets pre-load/post-load ads in the video - shall we support additional statuses for it?
• Playlists. Should we have a separate spec for them or just try to fit into the main one? I assume that each video in the playlist can just fire its own events, starting from play. If we do have playlists, we could cover image galleries and slideshows as well with next and prev or anything. But, perhaps, indeed, it will be easier to have a separate thread for it.

General:

• I'd deviate from HTML Media Element with regards to getVideoWidth and getVideoHeight. We'd use the same function for other embeds use cases (rels), and so, say, getSizes may be a better method. I've create a separate ticket for it Autoresize iframes spec #2 - it merges conversations from autoresize iframes spec iframely#29 too.
• More important, from the consumer apps standpoint, it is critical to be sure that if parent gives direction pause to iFrame, the video playback will stop. So we either need a confirmation back from iFrame (too complicated), or we need an initial handshake, which will help parent to discover the capabilities of iFrame embed and act accordingly (for example, replace iFrame'd video with image thumbnail while user watches other videos). I opened Initial parent-iframe handshake #3 for it. We might discuss password-protection or any paywall mechanisms there as well.

[iparamonau]

Oh, and let's not forget about full screen. I heard you saying many devs have asked to put it in one by default. There's a reason for it. People want next video to play in full screen if the prev one in their playlist was in full screen.

[iparamonau]

To keep everyone posted about the progress: Embedly guys came up with similar spec - https://github.com/embedly/player.js/blob/master/SPEC.rst

Let's see if we can unify.