Getting Control Center to show Spatial Audio toggle when using SFBAudioEngine.AudioPlayer

[kushalpandya]

I'm utilizing SFBAudioEngine.AudioPlayer in my music player app by creating a wrapper on top of it with some basic functions that my app needs (available to take a look here). I'm trying to get Spatial Audio control support such that macOS menubar starts showing a UI for it like it does in case of Apple Music (and also some other players);

From what I read so far (and looking at WWDC21 Session), is that this is possible when a playing source registers itself as MediaSession and more specifically, by playing audio via AVPlayer or AVSampleBufferAudioRenderer, but I could be wrong (as I'm new to SwiftUI and CoreAudio concepts in general). So any help in this direction would be useful!

[sbooth]

This is a great question that has led me down quite a few rabbit holes!

It does seem that in general integrating with the newer audio technologies requires the use of a system audio player.

I'm not really familiar with Spatial Audio but based on the WWDC session you linked it seems like AVSampleBufferAudioRenderer would be the most viable option to pursue, at least if you'd like to use any of the SFBAudioEngine components. I imagine that you'd basically throw out the player portion of SFBAudioEngine and use the decoders to read the various supported formats as PCM to hand off to the AVSampleBufferAudioRenderer. Apple has a code sample called SampleBufferPlayer that seems comprehensive but isn't exactly straightforward to understand.

[kushalpandya]

Thanks @sbooth! I suspected that passing PCM to AVSampleBufferAudioRenderer would be the only possible approach here. Also, Apple Engineering suggested in their developer thread the following which is interesting;

Using AVAudioEnvironmentNode will not enable the Spatial Audio options in Control Center, but that doesn't mean audio is not spatial.

So while we can go down the road of working SFBAudioEngine-provided PCM directly to get system-level toggle, alternatively, we can provide our own Spatial Toggle within the app while continuing to use SFBAudioEngine.AudioPlayer, pairing it with AVAudio3DPoint based on user setting.

Too bad that AVAudioEngine (being significantly powerful) doesn't get the feature by default and AVPlayer on the other hand doesn't support setting up custom decoders directly.