Audio stops working completely with random drops before

[fheinonen]

Describe the bug

I'm running a pipewire loopback setup sending audio from a apple tv to hdmi spdif splitter to UR23 spdif to usb converter. Then looping back to Schiit modi 3 on USB in pipewire configs. Also another device sending rtp signal to the pi and looping back to Schiit modi 3. The issue is that audio has random dropouts as a small snapping sound every 10-15 minutes or so happens way more on the spdif route. Maybe an hour later completely failing and only a full reboot fixes the issue. Changing pipewire or alsa settings made no difference. Also in earlier kernel before 6.12 i had no issues with these setup on bookworm. One reason i upgraded from bookworm was because there was this glitching that started happening, alas they are both 6.12.

What made the issue more tolerable was that i had Docker running traefik and homebridge which I disabled completely. With these running the dropout glitches would happen every 10-30 seconds and audio dying maybe 30 minutes later with a full reboot needed to restore audio.

This debian kernel mailing list issue describes this issue in great detail and i have the almost the same errors when audio fails completely. The symptoms are exactly the same. The ineffective steps mentioned in the first message i have also tried. The pastebin report doesn't have the callback suppressions yet, but here below in the logs is from an earlier failure
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1108496

Steps to reproduce the behaviour

1. Setup a pipewire loopback with a source and a sink in ~/.config/pipewire/pipewire.conf.d/30-loopback-spdif-to-dac.conf

context.modules = [
{
name = libpipewire-module-loopback
args = {
capture.props = {
node.name = "Loopback_Input"
node.target = "alsa_input.usb-HiFimeDIY_Audio_UR23_USB_SPDIF_Rx-01.iec958-stereo"
}
playback.props = {

node.name = "Loopback_Output"

node.target = "alsa_output.usb-Schiit_Audio_Schiit_Modi_3-00.analog-stereo"
}
}
}
]

Device (s)

Raspberry Pi 3 Mod. B

System

https://pastebin.com/qMYQZf5T

Logs

[23940.484943] retire_capture_urb: 4994 callbacks suppressed

[23940.930320] Transfer to device 4 endpoint 0x2 frame 1597 failed - FIQ reported NYET. Data may have been lost.

[23945.488945] retire_capture_urb: 4994 callbacks suppressed

[23950.492969] retire_capture_urb: 4994 callbacks suppressed

[23951.137316] Transfer to device 4 endpoint 0x2 frame 1564 failed - FIQ reported NYET. Data may have been lost.

[23955.496943] retire_capture_urb: 4994 callbacks suppressed

Additional context

No response

[pelwell]

The bug report at the link says that reading hwrng provokes the problem. Does running sudo dd if=/dev/hwrng of=/dev/null bs=64 count=1 trigger a glitch in your system?

[fheinonen]

No it does not, but the system symptoms are exactly the same.

[fheinonen]

Here is 100 lines of usbmon during the error when audio has stopped working with the current parameters. I had put dwc_otg.fiq_enable=0, to see if it made a diifference. Here's also a fresh pastebin with the audio not working.
https://pastebin.com/mZ536VvG
100linesusbmon.log

cat /proc/cmdline
coherent_pool=1M 8250.nr_uarts=0 snd_bcm2835.enable_headphones=0 cgroup_disable=memory snd_bcm2835.enable_headphones=1 snd_bcm2835.enable_hdmi=1 snd_bcm2835.enable_hdmi=0  vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyS0,115200 console=tty1 root=PARTUUID=34ed4db1-02 rootfstype=ext4 fsck.repair=yes dwc_otg.fiq_enable=0 rootwait

[P33M]

The parameters are a bit funky - to disable the FIQ codepaths, you need both dwc_otg.fiq_enable=0 dwc_otg.fiq_fsm_enable=0. The FIQ has no real beneficial effect on a 64-bit kernel as it runs in IRQ context.

I'd be interested if the behaviour is better/worse with dwc2 (upstream driver) instead of dwc_otg. In /boot/firmware/config.txt under a [pi3] or [all] section, add dtoverlay=dwc2,dr_mode=host and reboot.

Alternatively you can run 32-bit Raspberry Pi OS on a Pi 3, which will get you real FIQ context, but it might be incompatible with your use-case (docker).

[fheinonen]

Hi
I will try that on trixie. I had a bookworm sd card to try and downgraded to kernel 6.6 from repo. barely any errors or audio blips for over 24h hours and it still works after that time, unlike with 6.12 and lots of errors and audio blips the only difference being the kernel version. Bookworm kernel configs were default configs.

[fheinonen]

Tried using dtoverlay=dwc2,dr_mode=host in config.txt. The outcome was even worse. After ten minutes it started cutting out and then failed completely and even after failing completely i can hear audio pops every 30 seconds or so. Also CPU usage increased from 7-10% -> 14%.

I was pulling down with scp some usb logs a few megabytes worth. Then the audio suddenly started working during the file transfer and then after the transfer was done the audio was gone again. If did it again after a few minutes audio started working for a few minutes and then gone again. I managed to replicate this a few times. I have a usbmon log of that when it was working during the transfer. The usbmon log is a few megabytes so quite a few lines.

fullusbmon_transfer.log

https://pastebin.com/44U2XZdM

[fheinonen]

Additional thing I noticed about dwc2. When I came back 18h later and turned my apple tv on and audio was working flawlessly to my surprise. Now based on yesterdays findings I then logged in via SSH to the pi to see if it would have any effect and then audio cut out immediately. I then did a few scp transfers to get it back working. After about 5 transfers it started working again and logged out.