Spatial Sound

Please read the Spatial Sound and Spatial Sound in Unity documentation for an overview of the concept and Unity Spatial Sound settings.

If you've ever played Marco Polo, or had someone call your phone to help you locate it, you are already familiar with the importance of Spatial Sound. We use sound cues in our daily lives to locate objects, get someone's attention, or get a better understanding of our environment. The more closely your app's sound behaves like it does in the real world, the more convincing and engaging your holograms will be. Spatial Sound does four key things for holographic development:

1. Grounding: Just like real objects, you want to be able to hear holograms even when you can't see them, and you want to be able to locate them anywhere around you. Just as holograms need to be grounded visually to blend with your real world, they also need to be grounded audibly. Spatial Sound seamlessly blends your real world audio environment with the holographic audio environment.

2. User Attention: People are used to having their attention drawn by sound - we instinctually look toward an object that we hear around us. When you want to direct your user's gaze to a particular place, rather than using an arrow to point them visually, placing a sound in that location is a very natural and fast way to guide them.

3. Immersion: When objects move or collide, we usually hear those interactions between materials. So when your objects don't make the same sound they would in the real world, a level of immersion is lost - like watching a scary movie with the volume all the way down. Spatialized sound make up the "feel" of a place beyond what we can see.

4. Interaction Design: In most traditional interactive experiences, interaction sounds like UI sound effects are played in standard mono or stereo. But because everything in HoloLens exists in 3D space - including the UI - these objects benefit from spatialized sounds. When we press a button in the real world, the sound we hear comes from that button. By spatializing interaction sounds, we again provide a more natural and realistic user experience.

A few best practices when using Spatial Sound:

1. Real sounds work better than synthesized or unnatural sounds. The more familiar your user is with a type of sound, the more real it will feel, and the more easily they will be able to locate it in their environment. A human voice, for example, is a very common type of sound, and your users will locate it just as quickly as a real person in the room talking to them.

2. Expectation trumps simulation. If you are used to a sound coming from a particular direction, your attention will be guided in that direction regardless of spatial cues. For example, most of the time that we hear birds, they are above us. Playing the sound of a bird will most likely cause your user to look up, even if you place the sound below them. This is usually confusing, and it is recommended that you work with expectations like these rather than going against them for a more natural experience.

3. Most sounds should be spatialized. As mentioned above, everything in HoloLens exists in 3D space - your sounds should as well. Even music can sometimes benefit from spatialization, particularly when it's tied to a menu or some other UI.

4. Avoid invisible emitters. Because we've been conditioned to look at sounds that we hear around us, it can be an unnatural and even unnerving experience to locate a sound that has no visual presence. Sounds in the real world don't come from empty space, so be sure that if an audio emitter is placed within the user's immediate environment that it can also be seen.

5. Avoid spatial masking. Spatial Sound relies on very subtle acoustic cues that can be overpowered by other sounds. If you do have stereo music or ambient sounds, make sure they are low enough in the mix to give room for the details of your spatialized sounds that will allow your users to locate them easily, and keep them sounding realistic and natural.