Transcription mode is very unreliable

[thirdnull]

Checks

• I have searched existing issues and discussions
• I can reproduce this with the latest main or release

Describe the bug

Problem

Transcription mode is currently unreliable and difficult to use. The root cause appears to be the "type-as-you-speak" implementation. Streaming transcription output into the active text field in real time is an extremely hard problem to get right.

Suggested Fix

Replace the live-typing approach with a simpler push-to-talk model:

1. User presses a hotkey to start recording
2. User presses it again to stop
3. The transcribed text is copied to the clipboard and pasted into the active field (restoring the previous clipboard contents afterward)

Prior Art / Reference

SuperWhisper is a macOS app whose entire product is voice transcription, and it uses this exact pattern while deliberately avoiding live typing. Notably, it ships with the same Parakeet model Osaurus uses. If a dedicated transcription app considers live typing out of scope, it's a strong signal that the complexity isn't worth it for a secondary feature here.

Why This Matters

The clipboard approach is simpler to implement, significantly more reliable, and is already a proven UX pattern. The current implementation risks making transcription mode feel broken, which reflects poorly on the product overall. Scoping down to the clipboard model would ship something that actually works.

Steps to reproduce

No response

Provider type

Local model (MLX)

Model name

No response

Osaurus version / commit

Latest

macOS version

26.1

Apple Silicon chip

No response

Xcode version

No response

Logs

Screenshots

No response

[RaajeevChandran]

Hey. That's a fair point. Would having a toggle in the Voice settings that lets you change this behavior between real time transcription/fully processed text after recording sound like a solution?

[thirdnull]

@RaajeevChandran A toggle could work, though to be blunt, I wouldn't consider the current real-time mode shippable as-is regardless. The main issue is that it breaks down quickly if you pause while speaking. I use voice transcription a lot for notes and messages, and pausing to gather my thoughts mid-sentence is just part of how I talk. Neither macOS nor Windows native transcription has a problem with this, so it's become an ingrained habit. When Osaurus produces broken output the moment there's a brief silence, it's a pretty jarring regression from what users are used to.

So yes to a toggle, but the real-time mode would need to be stable enough to be worth offering in the first place.

[RaajeevChandran]

I completely agree that the real-time mode isn't polished yet and that's why it's disabled by default in the latest build (you can try it now). Removing real-time mode completely does not seem right, since existing users are now accustomed to it. A toggle to enable/disable it seemed to be a good middle ground.

And as for handling pauses, the latest build still assumes that you're done talking after a few moments of silence. But you do raise a valid concern so I'll create another toggle in the settings that let's you control when osaurus should stop recording the audio (Manual/Automatic). Automatic = listens for silence to stop recording and Manual = waits for your action (i.e. clicking the stop button)

[m13v]

the push-to-talk clipboard approach is definitely the right call here. we went through the same evolution building voice input for a macOS app - tried live streaming transcription first, and it breaks down exactly like you describe when users pause mid-sentence.

what worked for us: hotkey starts recording, hotkey stops, then we run WhisperKit inference on the full audio chunk. the transcription quality is way better on complete utterances vs streaming partial results. we do the clipboard paste thing too, works reliably across basically every app.

one detail that helped with the pause problem specifically - we added silence detection (simple RMS threshold) so if someone pauses for 3+ seconds it auto-segments without stopping the recording. that way you get natural paragraph breaks without the model getting confused by long silence gaps in the middle of audio.

[m13v]

our transcription service with the full push-to-talk pipeline, model management, and audio chunking: https://github.com/m13v/fazm/blob/main/Desktop/Sources/TranscriptionService.swift

has the silence detection, WhisperKit model lifecycle (download/load/inference), and the buffering logic for handling pauses.

[thirdnull]

@m13v Thanks for sharing your experience and the fazm source. Really useful reference point, especially the silence detection approach.

Building on that: if transcription mode is a feature we want to fully invest in, I'd recommend setting up a proper DSP chain on the audio input. At bare minimum, I'd consider a noise gate (along the lines of what m13v described with the RMS threshold) and a compressor. The gate keeps silence and background noise from confusing the model, and the compressor evens out volume so you get a more consistent signal going into inference regardless of how close someone is to the mic or how much their voice level varies. Those two together would go a long way toward making the transcription input cleaner before it ever hits the model.

Speaking as someone with experience working with audio, this is pretty standard signal chain stuff, but it seems like most transcription frameworks out there aren't really doing it.

[m13v]

great point on the DSP chain. we found that just RMS thresholding catches most silence but misses edge cases like keyboard clicks or fan noise that still trigger partial transcriptions. a proper noise gate with attack/release tuning would handle those much cleaner. have you benchmarked the latency impact of adding a compressor in the audio pipeline? curious if it adds noticeable delay for real-time use cases.