Skip to content

Commit 6ae0013

Browse files
gabrielsanbritogabrielsanbrito
authored
[WebAudio] AudioContext interrupted state (#854)
* add interrupted state explainer * Changed main README file * Add authors github and fix endlines * Addressing PR feedback * Address review feedback
1 parent 2cd01ce commit 6ae0013

File tree

2 files changed

+215
-0
lines changed

2 files changed

+215
-0
lines changed

‎AudioContextInterruptedState/explainer.md

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
# AudioContext Interrupted State
2+
3+
## Authors:
4+
5+
- [Gabriel Santana Brito](https://github.com/gabrielsanbrito)
6+
- [Steve Becker](https://github.com/SteveBeckerMSFT)
7+
8+
## Participate
9+
- https://github.com/WebAudio/web-audio-api/issues/2392
10+
11+
## Introduction
12+
13+
The [Web Audio API](https://webaudio.github.io/web-audio-api/) is widely used to add advanced audio capabilities to web applications, like web-based games and music applications. One of the API's features is the [AudioContext interface](https://webaudio.github.io/web-audio-api/#AudioContext), which represents an audio graph. An `AudioContext` can find itself in one of three states: [`"suspended"`](https://webaudio.github.io/web-audio-api/#dom-audiocontextstate-suspended), [`"running"`](https://webaudio.github.io/web-audio-api/#dom-audiocontextstate-running), or [`"closed"`](https://webaudio.github.io/web-audio-api/#dom-audiocontextstate-closed). Once an `AudioContext` is in the `"running"` state, it can only pause media playback by transitioning to the `"suspended"` state when, and only when, user code calls [`suspend()`](https://webaudio.github.io/web-audio-api/#dom-audiocontext-suspend) on this `AudioContext`. However, there are situations where we might want to let the User Agent (UA) decide when to interrupt playback - e.g., the proposed [`"media-playback-while-not-visible"`](https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/IframeMediaPause/iframe_media_pausing.md) permission policy, the proposed [Audio Session API](https://w3c.github.io/audio-session/), or during a phone call where the calling application will need exclusive access to the audio hardware. To support these scenarios, we propose adding a new `"interrupted"` state to the [`AudioContextState`](https://webaudio.github.io/web-audio-api/#enumdef-audiocontextstate) enum.
14+
15+
## Goals
16+
17+
The main goal of this proposal is to allow the UA to be able to interrupt `AudioContext` playback when needed, given that there are a couple of user scenarios - e.g., incoming phone calls, screen lock, etc - and proposed web API's that could make good use of this functionality.
18+
19+
### "media-playback-while-not-visible" permission policy
20+
21+
The ["media-playback-while-not-visible" permission policy](https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/IframeMediaPause/iframe_media_pausing.md) allows the UA to pause media playback on unrendered iframes. However, since the Web Audio API [does not allow the UA to suspend](https://webaudio.github.io/web-audio-api/#dom-audiocontext-suspended-by-user-slot) audio playback, the proposed permission policy has no mechanism available to pause media playback.
22+
23+
### Audio Session API
24+
25+
Similarly, the [Audio Session API](https://w3c.github.io/audio-session/) could also make use of the `"interrupted"` state. Whenever the [`AudioSessionState`](https://w3c.github.io/audio-session/#enumdef-audiosessionstate) is `"interrupted"`, the AudioContext of the document would also transition to the `"interrupted"` state. As a matter of fact, the `AudioContext`'s `"interrupted"` state has been [implemented on WebKit](https://github.com/WebKit/WebKit/blob/1e8ea6e4777297ce82e6c911caa7cce2cc32e6a9/Source/WebCore/Modules/webaudio/AudioContextState.idl) and is currently being used by the Audio Session API.
26+
27+
In Safari, on MacOS Sonoma 14.5, if there an active web page with the `AudioContext` in the `"running"` state, and if we set the audio session type to `"auto"` with `navigator.audioSession.type = "auto"`; whenever the laptop's screen is locked, the AudioContext will transition to the `"interrupted"` state. When the screen is unlocked, the state will automatically switch to `"running"` again.
28+
29+
### Exclusive access to audio hardware
30+
31+
There are scenarios where another application may acquire exclusive access to audio hardware. For example, when a phone call is in progress. In this situation, if there is already an `AudioContext` in the `running` state, it makes sense that the UA pauses the `AudioContext` using the `interrupted` state while the call is in progress.
32+
33+
## The `"interrupted"` state
34+
35+
This explainer proposes adding the `"interrupted"` state to the `AudioContextState` enum, as shown below:
36+
37+
```js
38+
enum AudioContextState {
39+
"suspended",
40+
"running",
41+
"closed",
42+
"interrupted"
43+
};
44+
```
45+
46+
Whenever an `AudioContext` transitions to either `"suspended"` or `"interrupted"`, audio playback would halt. The main difference between `"suspended"` and `"interrupted"` is that an `AudioContext` can only move to the `"suspended"` state if the [user triggered](https://webaudio.github.io/web-audio-api/#dom-audiocontext-suspended-by-user-slot) the state change; while the UA can transition the context to the `"interrupted"` state if there is a need for that.
47+
48+
With the addition of the `"interrupted"` state, the following state transitions would also be introduced:
49+
- `"running"` -> `"interrupted"`;
50+
- Would happen whenever the UA needs to interrupt audio playback.
51+
- `"suspended"` -> `"interrupted"`;
52+
- Shouldn't happen automatically. This transition should happen only if there is an ongoing interruption and [`AudioContext.resume()`](https://webaudio.github.io/web-audio-api/#dom-audiocontext-resume) is called.
53+
- `"interrupted"` -> `"running"`;
54+
- By the time that the cause of the interruption ceases to exist, the UA can transition to `"running"` if audio playback is allowed to resume automatically.
55+
- `"interrupted"` -> `"suspended"`; and
56+
- By the time that the cause of the interruption ceases to exist, the UA can transition to `"suspended"` if audio playback is **not** allowed to resume automatically or if [`AudioContext.suspend()`](https://webaudio.github.io/web-audio-api/#dom-audiocontext-suspend) has been called during the interruption.
57+
- `"interrupted"` -> `"closed"`.
58+
- The `AudioContext`'s state should move immediately to `"closed"` when `AudioContext.close()` is called.
59+
60+
The state transition from `"suspended"` to `"interrupted"` should not happen automatically, due to privacy concerns. Doing this transition automatically every time an interruption occurs might unnecessarily expose too much information to web pages - e.g., when a phone call comes in. To prevent fingerprinting in this scenario, an internal boolean flag, let's say `"is_interrupted"`, should be set and the AudioContext should remain in the `"suspended"` state, even though it is interrupted behind the curtains. However, calling [`AudioContext.resume()`](https://webaudio.github.io/web-audio-api/#dom-audiocontext-resume) should trigger the state change from `"suspended"` to `"interrupted"` if the interruption is still active. Therefore, while in the `"suspended"` state with an ongoing interruption:
61+
- Calling `AudioContext.resume()` would return a rejected promise and transition to the `"interrupted"` state;
62+
- Calling `AudioContext.suspend()` is a NOOP;
63+
64+
Finally, while in the `"interrupted"` state:
65+
- Calling `AudioContext.resume()` would return a rejected promise;
66+
- Calling `AudioContext.suspend()` would change the `AudioContext`'s state to `"suspended"` and return a promise that should resolve when the `AudioContext`'s state has transitioned to `"suspended"`.
67+
- Calling `AudioContext.close()` would return a promise that should resolve when the `AudioContext`'s state has transitioned to `"closed"`
68+
69+
## Key scenarios
70+
71+
The `"interrupted"` state can be used by a couple of proposed API's, like the [`"media-playback-while-not-visible"`](https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/IframeMediaPause/iframe_media_pausing.md) permission policy and the [Audio Session API](https://w3c.github.io/audio-session/) (this list is not exhaustive).
72+
73+
### "media-playback-while-not-visible" permission policy
74+
75+
Whenever an iframe, which has the `"media-playback-while-not-visible"` permission policy enabled, is not rendered anymore, the UA would transition the iframe's AudioContext to the `"interrupted"` state to pause audio rendering.
76+
77+
In the example below, we should have the following behavior:
78+
1. When the iframe is loaded, the application should initially print `"suspended"` on the console.
79+
2. Clicking on the "play-audio-btn" button, should start the AudioContext and `"running"` should be printed on the console.
80+
3. Clicking on the "iframe-visibility-btn" button, should hide the iframe and `"media-playback-while-not-visible"` will interrupt the AudioContext. Thus, `"interrupted"` should be printed on the console.
81+
4. Clicking again on the "iframe-visibility-btn" button, should show the iframe. The `"media-playback-while-not-visible"` permission policy will end the interruption and move the AudioContext to the `"running"` state. As a result, `"running"` should be printed on the console.
82+
83+
```html
84+
<!-- index.html -->
85+
<!DOCTYPE html>
86+
<html>
87+
<body>
88+
<button id="iframe-visibility-btn">Hide iframe</button>
89+
<div>
90+
<iframe id="audiocontext-iframe" src="audiocontext-iframe.html" allow="media-playback-while-not-visible 'none'; autoplay *"></iframe>
91+
</div>
92+
<script>
93+
const HIDE_IFRAME_BTN_STR = "Hide iframe";
94+
const SHOW_IFRAME_BTN_STR = "Show iframe";
95+
96+
const iframe_visibility_btn = document.getElementById("iframe-visibility-btn");
97+
98+
iframe_visibility_btn.addEventListener("click", () => {
99+
const audiocontext_iframe = document.getElementById("audiocontext-iframe");
100+
if (iframe_visibility_btn.textContent === HIDE_IFRAME_BTN_STR) {
101+
// Hide the iframe
102+
audiocontext_iframe.style.setProperty("display", "none");
103+
iframe_visibility_btn.textContent = SHOW_IFRAME_BTN_STR
104+
} else {
105+
// Show the iframe
106+
audiocontext_iframe.style.setProperty("display", "block");
107+
iframe_visibility_btn.textContent = HIDE_IFRAME_BTN_STR
108+
}
109+
});
110+
</script>
111+
</body>
112+
</html>
113+
```
114+
115+
```html
116+
<!-- audiocontext-iframe.html -->
117+
<!DOCTYPE html>
118+
<html>
119+
<body>
120+
<button id="play-audio-btn">Play audio</button>
121+
<script>
122+
const audio_context = new AudioContext();
123+
const oscillator = audio_context.createOscillator();
124+
oscillator.connect(audio_context.destination);
125+
126+
const play_audio_btn = document.getElementById("play-audio-btn");
127+
128+
audio_context.addEventListener("statechange", () => {
129+
console.log(audio_context.state);
130+
});
131+
132+
play_audio_btn.addEventListener("click", () => {
133+
oscillator.start();
134+
});
135+
136+
window.addEventListener("load", () => {
137+
console.log(audio_context.state);
138+
});
139+
</script>
140+
</body>
141+
</html>
142+
```
143+
144+
### Audio Session API
145+
146+
Whenever the Audio Session API needs to pause media playback, the document's active AudioContext would transition to the `"interrupted"` state. Let's say that a UA decides that the Web Audio API should not play any audio whenever the screen gets locked and the navigator's object [`AudioSessionType`](https://w3c.github.io/audio-session/#enumdef-audiosessiontype) is set to `"auto"`. Since the current Web Audio API spec does not allow the UA to transition the `AudioContext`'s state to `"suspended"`, the UA can instead move the `AudioContext` to the `"interrupted"` state. Once the interruption ends, if the UA does not resume playback automatically, the application code can monitor the `AudioContext`'s state changes and be able to call `AudioContext.resume()` when the `AudioContext` is in the `"suspended"` state.
147+
148+
Given the snippet below, where the `AudioSession` type is set to `"auto"`, we would have the following behavior:
149+
1. Page is loaded and the AudioContext is `"suspended"`.
150+
2. After clicking on the "Play Audio" button, the `AudioContext` state will be `"running"`.
151+
3. The screen gets locked. At this moment, based on the the Audio Session API type, the UA should interrupt the `AudioContext`, and the context's state should now be `"interrupted"`.
152+
4. The screen gets unlocked. The UA should lift the interruption and the `AudioContext` should transition to the `"running"` state.
153+
154+
```js
155+
<!DOCTYPE html>
156+
<html>
157+
<body>
158+
<button id="play-audio-btn">Play audio</button>
159+
<script>
160+
const audio_context = new AudioContext();
161+
const oscillator = audio_context.createOscillator();
162+
oscillator.connect(audio_context.destination);
163+
164+
const play_audio_btn = document.getElementById("play-audio-btn");
165+
166+
audio_context.addEventListener("statechange", () => {
167+
console.log(audio_context.state);
168+
});
169+
170+
play_audio_btn.addEventListener("click", () => {
171+
oscillator.start();
172+
});
173+
174+
window.addEventListener("load", () => {
175+
navigator.audioSession.type = "auto";
176+
console.log(audio_context.state);
177+
});
178+
</script>
179+
</body>
180+
</html>
181+
```
182+
183+
## Web compatibility risks
184+
185+
When `AudioContext.resume()` is called for an `AudioContext` in the `"closed"` state, the returned promise is rejected. With this proposal, the same behavior will happen when `AudioContext.resume()` is called while the `AudioContext` interrupted (could happen either during `"suspended"` or `"interrupted"`). In this case, a web page that is not aware of the existence of the "`interrupted`" state might think that the `AudioContext` has been closed.
186+
187+
## Privacy considerations
188+
189+
Moving the `AudioContexts` to the `"interrupted"` state might giveaway some information about the user's behavior - for example, when the user started a voice call or locked the screen. However, since "interrupting" an `AudioContext` could be associated with several types of interruptions, exposing this new state shouldn't be a major concern. Moreover, to mitigate privacy concerns, this proposal forbids automatically transitioning from `"suspended"` to `"interrupted"` to avoid unnecessarily exposing interruptions. This state transition will only happen if [`AudioContext.resume()`](https://webaudio.github.io/web-audio-api/#dom-audiocontext-resume) is called.
190+
191+
## Considered alternatives
192+
193+
This sections lists a number of alternatives taken into consideration prior to and during the writing of this document.
194+
195+
### Re-use the `"suspended"` state
196+
197+
We first considered in the [`"media-playback-while-not-visible"`](https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/IframeMediaPause/iframe_media_pausing.md) permission policy proposal using the `"suspended"` state whenever media playback needed to be paused. However, this is not possible, because [only user code can suspend](https://webaudio.github.io/web-audio-api/#dom-audiocontext-suspended-by-user-slot) an `AudioContext`.
198+
199+
## Stakeholder Feedback / Opposition
200+
201+
- Chromium : Positive
202+
- Chromium engineers have shown support for this feature in the [github WebAudio repository](https://github.com/WebAudio/web-audio-api/issues/2392#issuecomment-844465877).
203+
- WebKit : Positive
204+
- This feature is shipped in Safari:
205+
- [[MDN] Resuming interrupted play states in iOS Safari](https://developer.mozilla.org/en-US/docs/Web/API/BaseAudioContext/state#resuming_interrupted_play_states_in_ios_safari)
206+
- [[iOS] When Web Audio is interrupted by a phone call, it cannot be restarted.](https://github.com/WebKit/WebKit/commit/c2e380f844e9bbb3afea4d9ca8213f11e56a7ec4)
207+
- Gecko : No signals
208+
209+
## References & acknowledgements
210+
211+
Many thanks for valuable feedback and advice from:
212+
- [Erik Anderson](https://github.com/erik-anderson)
213+
- [Rahul Singh](https://github.com/rahulsingh-msft)
214+
- [Sunggook Chue](https://github.com/sunggook)

‎README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ we move them into the [Alumni section](#alumni-) below.
7979
| [Split Tab Navigation](https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/SplitTab/NavigationInSplitTab/explainer.md) | <a href="https://github.com/MicrosoftEdge/MSEdgeExplainers/labels/Split%20Tab">![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/Split%20Tab?label=issues)</a> | [New Issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=xuzhengyi1995&labels=Split+Tab&template=split-tab.md&title=%Split+Tab%5D+Issue) | Web Applications |
8080
| [Set Default Audio Output Device](SetDefaultSinkId/explainer.md) | <a href="https://github.com/MicrosoftEdge/MSEdgeExplainers/labels/SetDefaultSinkId"> ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/SetDefaultSinkId?label=issues)</a> | [New issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=kyerebo&labels=SetDefaultSinkId&template=setDefaultSinkId.md&title=%5BSetDefaultSinkId%5D+Issue) | WebRTC |
8181
| [Handwriting attribute](Handwriting/explainer.md) | <a href="https://github.com/MicrosoftEdge/MSEdgeExplainers/labels/Handwriting"> ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/Handwriting?label=issues)</a> | [New issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=adettenb&labels=Handwriting&template=Handwriting.md&title=%5BHandwriting%5D+Issue) | HTML |
82+
| [AudioContext Interrupted State](https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/AudioContextInterruptedState/explainer.md) | <a href="https://github.com/MicrosoftEdge/MSEdgeExplainers/labels/AudioContext%20Interrupted%20State">![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/AudioContext%20Interrupted%20State?label=issues)</a> | [New Issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=gabrielbrito&labels=AudioContext+Interrupted+State&title=%5BAudioContext+Interrupted+State%5D+%3CTITLE+HERE%3E) | WebAudio |
8283

8384

8485
# Alumni 🎓

0 commit comments

Comments
 (0)