non-determinism in tests

[drammock]

Locally if I run pytest -k test_play_sound I get all 4 tests passing, but if I just run pytest they seem to always fail. The error (see details) is a bit cryptic as it originates in portaudio C code, but the TLDR is that it can't initialize the ALSA stream, which causes the Python traceback:

RuntimeError: Could not set up sound card 'HDA Intel PCH: ALC892 Analog (hw:0,0)' (index=0) via ALSA, 2 channels @ 44100 Hz

(that card exists, and is specified as "SOUND_CARD_NAME": "HDA Intel PCH: ALC892 Analog" in my JSON config)

Details

expyfun/stimuli/tests/test_stimuli.py ..Expression 'ret' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1736
Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1904
Expression 'PaAlsaStreamComponent_Initialize( &self->playback, alsaApi, outParams, StreamDirection_Out, NULL != callback )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2175
Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags, userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2839
FExpression 'ret' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1736
Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1904
Expression 'PaAlsaStreamComponent_Initialize( &self->playback, alsaApi, outParams, StreamDirection_Out, NULL != callback )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2175
Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags, userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2839
FExpression 'ret' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1736
Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1904
python3.13: src/hostapi/alsa/pa_linux_alsa.c:2178: PaAlsaStream_Initialize: Assertion `self->capture.nfds || self->playback.nfds' failed.
Fatal Python error: Aborted

I thought this was due to a recent OS change where I uninstalled portaudio in favor of using pipewire everywhere (to fix some issues I was having with my speakers and mic), but then I realized the tests pass when run in isolation, so I don't think that can be the cause. FWIW, installing portaudio into the mamba environment don't seem to help.

Relatedly, when run in isolation pytest -k test_ec[pyglet] and pytest -k test_ec[sounddevice] and pytest -k test_gapless[sounddevice] all pass, but all the EC tests fail if run as pytest -k test_ec and all three fail if run as just pytest. They give different python tracebacks, e.g.:

• pyglet.media.drivers.pulse.interface.PulseAudioException: PulseAudioException: [8] Timeout
• sounddevice.PortAudioError: Error opening OutputStream: Device unavailable [PaErrorCode -9985]

...but the same C lib traceback also appears, so I think it's all the same underlying cause.

A hunch is that there's some sort of concurrency lock or teardown happening in the audio libs, causing the ALSA stream initialization to fail.

[larsoner]

There was some problem in testing recently where switching backends didn't really seem possible... see

https://github.com/LABSN/expyfun/pull/494/files

But that was on Windows. Not sure if there is some similar problem on linux. Might be unrelated but it came to mind at least

A hunch is that there's some sort of concurrency lock or teardown happening in the audio libs, causing the ALSA stream initialization to fail.

Yeah maybe we need some longer wait somewhere to allow it enough time to give up a lock on the sound card or something. We do have this

expyfun/expyfun/tests/test_experiment_controller.py

Line 41 in 3f1ea96

SAFE_DELAY = 0.5 if platform.system() == "Windows" else 0.2

That maybe needs to be longer for you?

[drammock]

bumping SAFE_DELAY to 0.5 or 1.5 seconds is still not enough to get test_ec[sounddevice] to work for me (test_ec[pyglet] I temporarily worked around in #535). I'll dig into the sounddevice code later to see if I can figure out what's going on.