Skip to content

[CI] Enable audio tests in Chrome #23665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Feb 18, 2025
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -306,16 +306,18 @@ commands:
- run:
name: run tests (<< parameters.title >>)
environment:
EMTEST_LACKS_SOUND_HARDWARE: "1"
EMTEST_DETECT_TEMPFILE_LEAKS: "0"
# --no-sandbox because we are running as root and chrome requires
# this flag for now: https://crbug.com/638180
CHROME_FLAGS_BASE: "--no-first-run -start-maximized --no-sandbox --use-gl=swiftshader --user-data-dir=/tmp/chrome-emscripten-profile --enable-experimental-web-platform-features"
CHROME_FLAGS_HEADLESS: "--headless=new --remote-debugging-port=1234"
CHROME_FLAGS_WASM: "--enable-experimental-webassembly-features --js-flags=\"--experimental-wasm-stack-switching --experimental-wasm-type-reflection\""
CHROME_FLAGS_NOCACHE: "--disk-cache-dir=/dev/null --disk-cache-size=1 --media-cache-size=1 --disable-application-cache --incognito"
# The runners lack sound hardware so fallback to a dummy device (and
# bypass the user gesture so audio tests work without interaction)
CHROME_FLAGS_AUDIO: " --use-fake-device-for-media-stream --autoplay-policy=no-user-gesture-required"
command: |
export EMTEST_BROWSER="/usr/bin/google-chrome $CHROME_FLAGS_BASE $CHROME_FLAGS_HEADLESS $CHROME_FLAGS_WASM $CHROME_FLAGS_NOCACHE"
export EMTEST_BROWSER="/usr/bin/google-chrome $CHROME_FLAGS_BASE $CHROME_FLAGS_HEADLESS $CHROME_FLAGS_WASM $CHROME_FLAGS_NOCACHE $CHROME_FLAGS_AUDIO"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we make this change should we also remove the EMTEST_LACKS_SOUND_HARDWARE setting above?

I'm kind of inclined to make this change as its one separate PR, e.g. [CI] Enable audio testing on chrome?

Copy link
Contributor Author

@cwoffenden cwoffenden Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't have much time today so it was mostly experimenting and coming back to it, but:

  • I'll remove theEMTEST_LACKS_SOUND_HARDWARE
  • I'll rename to match its new purpose (I wasn't sure what I'd achieve when starting) ✅
  • I'll disable all the wasm64 and 2GB tests with audio for now

Is it fine to always enable the dummy sound hardware and bypass the media interaction? I can't think what this would break (and where real sound hardware is present its used as a preference, at least from my manual testing).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I imagine enabling dummy sounds hardware and bypass the interaction check should be fine for all out tests..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I limited removing EMTEST_LACKS_SOUND_HARDWARE to Chrome (FF I can't get to work, documented in the comments).

# There are tests in the browser test suite that using libraries
# that are not included by "./embuilder build ALL". For example the
# PIC version of libSDL which is used by test_sdl2_misc_main_module
Expand Down Expand Up @@ -360,6 +362,8 @@ commands:
user_pref("javascript.options.shared_memory", true);
user_pref("javascript.options.wasm_memory64", true);
user_pref("dom.postMessage.sharedArrayBuffer.bypassCOOP_COEP.insecure.enabled", true);
user_pref("media.navigator.streams.fake", true);
user_pref("media.autoplay.default", 0);
EOF
- run:
name: run tests (<< parameters.title >>)
Expand All @@ -369,11 +373,12 @@ commands:
# support in headless mode) resolves
EMTEST_LACKS_GRAPHICS_HARDWARE: "1"
EMTEST_LACKS_WEBGPU: "1"
EMTEST_LACKS_SOUND_HARDWARE: "1"
# OffscreenCanvas support is not yet done in Firefox.
EMTEST_LACKS_OFFSCREEN_CANVAS: "1"
EMTEST_DETECT_TEMPFILE_LEAKS: "0"
DISPLAY: ":0"
# Note: as per Chrome, the FF profile enables a dummy audio device
# (so doesn't need the EMTEST_LACKS_SOUND_HARDWARE var)
command: |
export EMTEST_BROWSER="$HOME/firefox/firefox -headless -profile $HOME/tmp-firefox-profile/"
# There are tests in the browser test suite that using libraries
Expand Down
7 changes: 3 additions & 4 deletions test/test_browser.py
Original file line number Diff line number Diff line change
Expand Up @@ -5457,7 +5457,7 @@ def test_full_js_library_strict(self):
# Tests the AudioWorklet demo
@parameterized({
'': ([],),
'memory64': (['-sMEMORY64'],),
# 'memory64': (['-sMEMORY64'],),
'with_fs': (['--preload-file', test_file('hello_world.c') + '@/'],),
'closure': (['--closure', '1', '-Oz'],),
'asyncify': (['-sASYNCIFY'],),
Expand All @@ -5469,10 +5469,9 @@ def test_full_js_library_strict(self):
'es6': (['-sEXPORT_ES6'],),
'strict': (['-sSTRICT'],),
})
# @requires_sound_hardware
def test_audio_worklet(self, args):
if '-sMEMORY64' in args and is_firefox():
self.skipTest('https://github.com/emscripten-core/emscripten/issues/19161')
self.btest_exit('webaudio/audioworklet.c', args=['-sAUDIO_WORKLET', '-sWASM_WORKERS'] + args)
self.btest_exit('webaudio/audioworklet.c', args=['-sAUDIO_WORKLET', '-sWASM_WORKERS', '-DTEST_AND_EXIT'] + args)

# Tests that audioworklets and workers can be used at the same time
def test_audio_worklet_worker(self):
Expand Down
38 changes: 27 additions & 11 deletions test/webaudio/audioworklet.c
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,17 @@
begin to fire.
*/

// REPORT_RESULT is defined when running in Emscripten test harness. You can
// strip these out in your own project.
#ifdef REPORT_RESULT
// TEST_AND_EXIT is defined when running in Emscripten test harness. You can
// strip these out in your own project (otherwise playback will end quickly).
#ifdef TEST_AND_EXIT
_Thread_local int testTlsVariable = 1;
int lastTlsVariableValueInAudioThread = 1;
#endif

// This function will be called for every fixed-size buffer of audio samples to be processed.
bool ProcessAudio(int numInputs, const AudioSampleFrame *inputs, int numOutputs, AudioSampleFrame *outputs, int numParams, const AudioParamFrame *params, void *userData) {
#ifdef REPORT_RESULT
#ifdef TEST_AND_EXIT
// Only running in the test harness, see main_thread_tls_access()
assert(testTlsVariable == lastTlsVariableValueInAudioThread);
++testTlsVariable;
lastTlsVariableValueInAudioThread = testTlsVariable;
Expand Down Expand Up @@ -63,14 +64,15 @@ EM_JS(void, InitHtmlUi, (EMSCRIPTEN_WEBAUDIO_T audioContext), {
};
});

#ifdef REPORT_RESULT
#ifdef TEST_AND_EXIT
bool main_thread_tls_access(double time, void *userData) {
// Try to mess the TLS variable on the main thread, with the expectation that
// it should not change the TLS value on the AudioWorklet thread.
// it should not change the TLS value on the AudioWorklet thread, asserted in
// ProcessAudio().
testTlsVariable = (int)time;
// Exit to the test harness after enough calls to ProcessAudio()
if (lastTlsVariableValueInAudioThread >= 100) {
REPORT_RESULT(0);
return false;
emscripten_force_exit(EXIT_SUCCESS);
}
return true;
}
Expand All @@ -79,7 +81,11 @@ bool main_thread_tls_access(double time, void *userData) {
// This callback will fire after the Audio Worklet Processor has finished being
// added to the Worklet global scope.
void AudioWorkletProcessorCreated(EMSCRIPTEN_WEBAUDIO_T audioContext, bool success, void *userData) {
if (!success) return;
if (!success) {
emscripten_out("Stopped in AudioWorkletProcessorCreated");
assert(0);
return;
}

// Specify the input and output node configurations for the Wasm Audio
// Worklet. A simple setup with single mono output channel here, and no
Expand All @@ -97,7 +103,8 @@ void AudioWorkletProcessorCreated(EMSCRIPTEN_WEBAUDIO_T audioContext, bool succe
// Connect the audio worklet node to the graph.
emscripten_audio_node_connect(wasmAudioWorklet, audioContext, 0, 0);

#ifdef REPORT_RESULT
#ifdef TEST_AND_EXIT
// Schedule this to exit after ProcessAudio() has been called 100 times
emscripten_set_timeout_loop(main_thread_tls_access, 10, 0);
#endif

Expand All @@ -108,7 +115,11 @@ void AudioWorkletProcessorCreated(EMSCRIPTEN_WEBAUDIO_T audioContext, bool succe
// AudioWorklet global scope, and is now ready to begin adding Audio Worklet
// Processors.
void WebAudioWorkletThreadInitialized(EMSCRIPTEN_WEBAUDIO_T audioContext, bool success, void *userData) {
if (!success) return;
if (!success) {
emscripten_out("Stopped in WebAudioWorkletThreadInitialized");
assert(0);
return;
}

WebAudioWorkletProcessorCreateOptions opts = {
.name = "noise-generator",
Expand All @@ -132,4 +143,9 @@ int main() {
// and kick off Audio Worklet scope initialization, which shares the Wasm
// Module and Memory to the AudioWorklet scope and initializes its stack.
emscripten_start_wasm_audio_worklet_thread_async(context, wasmAudioWorkletStack, sizeof(wasmAudioWorkletStack), WebAudioWorkletThreadInitialized, 0);

#ifdef TEST_AND_EXIT
// We're in the test harness and exiting is via main_thread_tls_access()
emscripten_exit_with_live_runtime();
#endif
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking or a separate PR to enable audio testing CI.

PR1: enable audio testing in CI
PR2: fix audio worklet test so they fail correctly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, NP. I’ll see how I get on with FF on Monday but I might split it for Chrome first too (I set up a VM with the same Debian and FF and so far, even with no audio device, it doesn’t fail; left to try is headless).

Copy link
Contributor Author

@cwoffenden cwoffenden Feb 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After getting an audio-less Debian VM set-up with all the fallback sound devices removed, I finally managed to recreate what Firefox does. It fails here playing manually, or the test times out otherwise:

audioContext.resume();

The promise calls neither the success nor failure route, it just silently fails and the audio remains permanently suspended. I'll call this a Firefox bug and keep the EMTEST_LACKS_SOUND_HARDWARE (I'm satisfied that I've spent enough time going down all avenues and what remains is creating a repro in minimal JS for Firefox).

I'll just enable audio testing in CI for Chrome for now.

}