Enhancement: Implement audio streaming #4657
Description
Issue Checklist
- I have read the Contributing Guide
- I have checked the Issues/Discussions pages to see if my enhancement has already been suggested
- I have properly titled my enhancement
What is your suggestion, and why should it be implemented?
Audio streaming is a technique that could be used to reduce memory usage and loading times on desktop targets. (See for example: #3718). Instead of loading the full audio data in order to play a song, Lime will automatically load and unload a chunk of audio data as its playing.
Using audio streaming is pretty simple:
// I believe this creates a handle to the audio file so we can read however many bytes
// at whatever position similar as to how Haxe's sys.io.File.read() method works.
var vorbisFile = lime.media.vorbis.VorbisFile.fromFile("path/to/some/song.ogg");
// This isn't actually a "real" audio buffer (as in it doesn't store any audio data), it just holds
// a reference to the VorbisFile so Lime knows to stream it when playing.
var buffer = lime.media.AudioBuffer.fromVorbisFile(vorbisFile);
// Nothing special with the rest of these steps, we just create an OpenFL Sound so we can pass it to FlxSound
var flashSound = Sound.fromAudioBuffer(buffer);
var sound = FlxG.sound.load(flashSound);It was implemented in FlxPartialSound with the commit FunkinCrew/FlxPartialSound@e22f939, but this was reverted in Funkin' because of crashes in 7a624bc (unfortunately no info about the cause).
IMO this should be handled in Funkin' instead of FlxPartialSound because it leads to inconsistent behavior. You'd expect FlxPartialSound to give you audio data for a segment of a sound but in this case it'd still be the whole sound. Maybe Funkin' should have a utility function or something to request a partial sound, where it either opens a VorbisFile (and sets FlxSound.loopTime and FlxSound.endTime) or asks FlxPartialSound for proper cropped audio data. In this case we could also reuse the same sound instance for both the Freeplay preview and the song played during gameplay.
Audio that gets hooked up to a visualizer will cause issues because the audio buffer data is nonexistant. A workaround would be to either read the full data from the stream (reference code from one of my projects) or avoid getting streamed and load the full audio data as done previously, although I'm sure the code in funkin.vis could be adapted to work with streamed sounds. I could probably look into that Will be fixed in funkin.vis with FunkinCrew/funkVis#13