Skip to content

Enable short fadeout when stopping playback to prevent artifacts#314

Merged
francamps merged 14 commits intomainfrom
franc/fade-interruption
May 6, 2025
Merged

Enable short fadeout when stopping playback to prevent artifacts#314
francamps merged 14 commits intomainfrom
franc/fade-interruption

Conversation

@francamps
Copy link
Copy Markdown
Contributor

@francamps francamps commented Apr 29, 2025

Context

When audio playback is stopped abruptly before the clip reaches its natural endpoint, it often results in a "clicking" sound. This occurs because the audio waveform is interrupted while at a high amplitude, creating a sudden discontinuity. The occurrence of this clicking sound is unpredictable, as it depends on the random point in the waveform at which playback is halted.

This happens frequently when using the voice SDK and the user interrupts the assistant, which requires the client to stop the playback as soon as it receives the user_interruption event.

Change

This PR enables introducing a short fadeout when interrupting the assistant, which prevents any clicking sounds or other potential artifacts by stopping the playback always at a point with amplitude set to virtually 0. The fadeout can be called when clearing the buffer in the worklet. The gain manipulation is applied in the worklet itself, which allows this manipulation to occur at the sample level. The alternative of applying it via a GainNode in the main thread was discarded, as it required estimating the fadeout duration by polling the currentTime of the audio clip or using a setTimeout.

The change is backwards compatible, as it preserves the previous clear message to the worklet, which does still clear the buffer in the worklet without applying a fadeout.

Notes

This change introduces a new nomenclature to worklets: when a change to the worklet is merged, we will rename to new version as audio-worklet-YYYYMMDD.

@vercel
Copy link
Copy Markdown
Contributor

vercel bot commented Apr 29, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
empathic-voice-api-js-next-app ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 6, 2025 5:42pm

@francamps francamps changed the title Use fade out with worklet after user interruption DO NOT MERGE Use fade out with worklet after user interruption May 2, 2025
@francamps francamps force-pushed the franc/fade-interruption branch 2 times, most recently from 2823c7b to 48546a2 Compare May 5, 2025 18:34
@francamps francamps force-pushed the franc/fade-interruption branch from 48546a2 to 0b5268d Compare May 5, 2025 18:38
@francamps francamps force-pushed the franc/fade-interruption branch from 0b5268d to cec4c28 Compare May 5, 2025 18:54
@francamps francamps force-pushed the franc/fade-interruption branch from d2abe1e to 97ecda2 Compare May 5, 2025 19:51
@francamps francamps force-pushed the franc/fade-interruption branch from 1558dc8 to 588917a Compare May 5, 2025 20:29
@francamps francamps changed the title DO NOT MERGE Use fade out with worklet after user interruption DO NOT MERGE Enable short fadeout when stopping playback to prevent artifacts May 6, 2025
Copy link
Copy Markdown
Contributor

@vpicone vpicone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds great to me. Can definitely here the elimination of the audio artifact compared to main.

@francamps francamps force-pushed the franc/fade-interruption branch from 492472a to 3be89e7 Compare May 6, 2025 17:41
@francamps francamps changed the title DO NOT MERGE Enable short fadeout when stopping playback to prevent artifacts Enable short fadeout when stopping playback to prevent artifacts May 6, 2025
@francamps francamps merged commit 6e952a8 into main May 6, 2025
3 checks passed
@francamps francamps deleted the franc/fade-interruption branch May 6, 2025 17:43
yinishi added a commit that referenced this pull request May 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants