Why the move to HTML5 audio

[meytin1337]

Hello, I would like to ask why you decided to fully move to HTML5 Audio in version 7. Isn't the web audio API way more powerful? I mean I have seen that there is a way to use the web audio API nevertheless but still.. I'm pretty new in the field and maybe there is something I don't understand so I would appreciate if you could elaborate. Thanks and regards.

[katspaugh]

Good question!
Short answer: memory and power consumption.

Web Audio

Web Audio (specifically a BufferSourceNode) as the default playback mechanism was originally picked because we already used Web Audio to decode the audio. The decoded buffer was just passed to a BufferSourceNode directly, and played.

Web Audio pros

The pros of this approach were:

• Very precise timing and scheduling
• You can add filters, analyzers etc
• The audio is downloaded only once

Web Audio cons

• The buffer source wasn't designed for large audio. Its goal in Web Audio is short audio clips for games and sound effects. It's resource-intensive with larger files.
• Memory leaks (might still be a problem with decodeAudioData which we still use)
• The timing code is complicated – a buffer node doesn't track its own time, you have to write that logic in JS. Web Audio only offers a high-precision global timer in the AudioContext itself. Playback start time, end time and currentTime have to be calculated relative to that.
• People usually have their own Web Audio graph, or want to control its different parameters. Wavesurfer.js having its own instance of AudioContext seems limiting. There was an option to pass your own AudioContext, but that feels like an afterthought (and it was in fact).

HTML5 Audio

Now, HTML5 audio also has downsides, but a lot of benefits too.

HTML5 Audio pros

• It doesn't drain your battery and memory
• It's natively optimized for simple audio playback
• Comes with its own timing (you can set and read currentTime)
• It's essentially a full-featured player so you don't need to write any playback code, it even has its own UI if desired
• It's still possible to use Web Audio features via MediaElementSource (as you correctly noticed)
• It can be a video player too

HTML5 Audio cons

The downsides are:

• The audio has to be (partially) fetched twice (unless you provide your own peaks) Not the case anymore, the audio is fetched only once
• Seeking can be wonky depending on the network conditions because it basically streams the audio in chunks (which is a good thing)
• "currentTime" is less precise

Conclusion

To conclude, we were getting quite a few complaints about the Web Audio performance/memory issues, and the answer was always "Use backend: "MediaElement"". I think this speaks for itself. IMHO, the media element API overall is a better fit for our player use case.
That said, I'm open to alternative suggestions and further discussions! Curious what you think.

[meytin1337]

Thanks for the comprehensive reply. I have read about BufferSourceNode not being made for longer audio tracks in the MDN docs before. Performance is actually an issue for me so I guess I will give version 7 another try.

[sundayz]

Are you going to remove functions like setVolume() from the public API? You’re aware that you can’t programmatically set the volume on HTMLMediaElement nodes on iOS Safari, right?

[katspaugh]

@sundayz setVolume will remain. Browser quirks have always existed, and we should aim to shim/fix what we can, or at least document what we can’t.