Implement VirtIO sound device playback #53

Cuda-Chen · 2024-08-31T08:17:47Z

Implement VirtIO sound device supporting these operations (the item with checked box checked means it is implemented right now):

For setting up the device:
- VIRTIO_SND_R_JACK_INFO
- VIRTIO_SND_R_PCM_INFO
- VIRTIO_SND_R_CHMAP_INFO
For playing the sound (PCM):
- VIRTIO_SND_R_PCM_SET_PARAMS
- VIRTIO_SND_R_PCM_PREPARE
- VIRTIO_SND_R_PCM_RELEASE
- VIRTIO_SND_R_PCM_START
- VIRTIO_SND_R_PCM_STOP

Limitations

The emulator will hang if PulseAudio is enabled on host.
It causes to CNFA and CNFA cannot initialize if PulseAudio
is disabled, What's worse, it needs re-working as CNFA uses
different threading models between pure ALSA and PulseAudio environment.
Therefore, users need mitigations (i.e., restart the emulator after
playing sound) or just wait for the future release.
The playback may plays with repeating artifact (for example,
A "front center" ALSA example sound will sound like "front front
center"). The root cause is the Linux Kernel and it hasn't got
fixed even in mainline version.

Test Cases

boot up test

test procedures

Execute make check to run semu.
Check kernel message (dmesg).

expected results

The following message should appear while booting up:

[    4.011962] ALSA device list:
[    4.015962]   #0: Loopback 1
[    4.015962]   #1: VirtIO SoundCard at platform/f4400000.virtio/virtio2

check driver configuration

test procedures

Execute aplay -l in emulator.
Check the output messages.

expected results

The following message should appear after Step 1:

$ aplay -l          
**** List of PLAYBACK Hardware Devices ****

<other sound device here>

card 1: SoundCard [VirtIO SoundCard], device 0: virtio-snd [VirtIO PCM 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

play sound

test procedures

Execute speaker-test in emulator.
Check the host speaker.

expected results

A white noise will be played by host speaker while speaker-test is executing.

play sequence sound

test procedures

Execute aplay /usr/share/sounds/alsa/Front_Center.wav in emulator.
Check the host speaker.

expected results

A sound with "Front Center" speaking should be played.

CNFA_sf.h

virtio-snd.c

configs/linux.config

os_generic.h

Makefile

configs/linux.config

virtio-snd.c

jserv

Run clang-format before submitting.

.gitmodules

Makefile

jserv

Consult https://github.com/cntools/cnfa/blob/master/.github/workflows/build-cnfa.yml and mention the build dependency in top-level README.md.

configs/linux.config

feature.h

README.md

configs/linux.config

jserv · 2025-01-22T05:11:09Z

If you would like to get the file, I will convert the file into base64 encoded text in order to upload to here.

Simply share the prebuilt Linux kernel image file, rootfs, and test files via Google Drive, Dropbox, or similar services as they are intended to be temporary.

Cuda-Chen · 2025-01-22T07:36:23Z

Hi @jserv ,

This Dropbox share link contains the files that reviewers will need: https://www.dropbox.com/scl/fo/vv3luiccfco5tjwt8tqxm/AANAyYgL9qvVPhuT4dtkkIA?rlkey=dv84qtw9y4e0ea8go03q5qwf8&st=73j9agts&dl=0

The usage of files are described as follows:

Image: guest Linux OS image.
rootfs.cpio: disk image.
out_stream.bin: a binary dump of output stream (i.e., the Front_Center.wav of ALSA sound example). You can use aplay to play the file like this: cat out_stream.bin | aplay --format=S16_LE --rate=48000 -.

jserv · 2025-01-23T09:10:44Z

Tested on macOS + Apple M1:

# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: SoundCard [VirtIO SoundCard], device 0: virtio-snd [VirtIO PCM 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
# aplay /usr/share/sounds/alsa/Front_Center.wav
Playing WAVE '/usr/share/sounds/alsa/Front_Center.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Mono
CoreAudio: from: [O/I] (null)/(null) (semu-virtio-snd) / (48000,0)x(1,0) (4800)

Then, it got stuck without progress.

Cuda-Chen · 2025-01-23T09:17:22Z

Then, it got stuck without progress.

Thanks for helping the tests on macOS. I guess it is some kind of locking issue.
While trying to fix, I would like to ask whether a "front center" sound comes out from host or not while executing aplay /usr/share/sounds/alsa/Front_Center.wav?

For comparison on Linux + X86 it runs like this:

# aplay /usr/share/sounds/alsa/Front_Center.wav 
Playing WAVE '/usr/share/sounds/alsa/Front_Center.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Mono
CNFA Alsa Init (nil) (nil)  (48000 0) 1 0
0x55f41abed1d0 (nil)
PERIOD: 4800  BUFFER: 14400
CNFA Alsa Init Out -> 0x55f41abed1d0 (nil)  (48000 0) 1 0
(... play "front front center" then stuck in aplay program)

jserv · 2025-01-23T09:54:19Z

Consider the change below to suppress compilation warnings raised by CNFA:

--- a/Makefile
+++ b/Makefile
@@ -85,6 +85,7 @@ CNFA_LIB := cnfa/CNFA_sf.h
 $(CNFA_LIB): cnfa/Makefile cnfa/os_generic
        $(MAKE) -C $(dir $<) CNFA_sf.h
 main.o: $(CNFA_LIB)
+virtio-snd.o: CFLAGS += -Wno-unused-parameter
 endif
 
 # .DEFAULT_GOAL should be set to all since the very first target is not all

Cuda-Chen · 2025-01-23T10:29:56Z

Hi @jserv ,

Then, it got stuck without progress.

After checking the CNFA part, the coreaudio just like the PulseAudio/PipeWire part: no need to use a dedicated thread for pushing PCM frames.

I would like to ask which one you prefer for resolving this issue:

Exclude macOS support.
- No need to change the threading model of pushing PCM.
Use PulseAudio/PipeWire solution.
- Threading module of pushing PCM frames need to be re-designed.

virtio-snd.c

jserv · 2025-02-01T12:44:49Z

I would like to ask which one you prefer for resolving this issue:

Exclude macOS support.

No need to change the threading model of pushing PCM.

It is reasonable to exclude macOS.

utils.h

jserv · 2025-02-01T15:50:59Z

utils.h

+
+#ifndef container_of
+#ifdef __LIST_HAVE_TYPEOF
+#define container_of(ptr, type, member)                            \


You don't have to check __LIST_HAVE_TYPEOF here.

utils.h

The limitations are listed as follows: 1. The emulator will hang if PulseAudio is enabled on host. The reason is that the host OS cannot close CNFA driver because CNFA driver reference count is not zero (PulseAudio holds one reference count). What's worse, CNFA cannot initialize if PulseAudio is disabled, As it needs a dedicated threading model (CNFA uses different threading models between pure ALSA and PulseAudio environment), we suggest users need mitigations (for instance, restart the emulator after playing sound) or just wait for the future release. 2. The playback may play with repeating artifact (for example, A "front center" ALSA example sound will sound like "front front center"). The root cause is the Linux Kernel and it hasn't got fixed even in mainline version. See https://lore.kernel.org/all/[email protected]/T/ for more information.

jserv · 2025-02-02T02:44:46Z

Thank @Cuda-Chen for contributing!

Cuda-Chen force-pushed the add-virtio-snd branch from 4a8a6fc to 0902227 Compare August 31, 2024 08:21

jserv requested a review from shengwen-tw August 31, 2024 08:59

This comment was marked as outdated.

Sign in to view

jserv reviewed Aug 31, 2024

View reviewed changes

CNFA_sf.h Outdated Show resolved Hide resolved

jserv reviewed Aug 31, 2024

View reviewed changes

virtio-snd.c Outdated Show resolved Hide resolved

Cuda-Chen force-pushed the add-virtio-snd branch from 0902227 to ebba2c7 Compare September 1, 2024 06:05