Skip to content

Commit 26e96a2

Browse files
feat(tts): searchable voice picker, voice selection end-to-end
Replace the Voice gender (Female/Male) and gender-filtered Voice selects in Settings → Voice with a single searchable VoicePicker (Popover + Command/cmdk) that lists all voices — 22 curated Azure Neural voices plus (in dev) 184 macOS system voices — with type-to-search filtering by name and id. - Remove ttsVoiceGender from AppPreferences and preferences IPC (default, load, validation, set); drop gender props from App/SettingsScreen/TtsPanel. - add ui/popover and ui/command primitives; deps cmdk, @radix-ui/react-popover. - Voice model (TtsVoiceSource, TtsVoice.gender) and voices.ts: AZURE_VOICES, parseSayVoices, listVoices (dev-only macOS enumeration), normalizeSelectedVoiceId, voiceIsAzure. Thread selected voice through createTtsProvider -> azure/say. - tts:list-voices + tts:test (independent TtsManager + audio output start/stop, fixes silent test and 'TTS is already running'). - tests/voices.test.ts (12 tests); docs updated. type-check, 80/80 tests, build, eslint, prettier all clean.
1 parent 25c9f99 commit 26e96a2

24 files changed

Lines changed: 1243 additions & 53 deletions

docs/ARCHITECTURE.md

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -355,12 +355,14 @@ from the environment:
355355

356356
- `azure` (default) — Azure Speech TTS, same credentials as STT
357357
(`AZURE_SPEECH_KEY` + `AZURE_SPEECH_REGION`). Uses `SpeechSynthesizer` from
358-
`microsoft-cognitiveservices-speech-sdk` (already a dependency). Configurable
359-
voice via `AZURE_TTS_VOICE` (default `en-US-JennyNeural`). Uses `null`
358+
`microsoft-cognitiveservices-speech-sdk` (already a dependency). Voice from
359+
the Settings selection (added 2026-09-03) beats `AZURE_TTS_VOICE`
360+
(default `en-US-JennyNeural`). Uses `null`
360361
AudioConfig to get raw PCM from `result.audioData` (no system speaker
361362
output); audio is routed through `AudioOutputManager`.
362363
- `say` — macOS built-in `say` command. Zero dependencies, fully offline.
363-
Uses `Samantha` voice at 200 wpm. `stop()` kills via `killall say`.
364+
Voice from the Settings selection adds `-v <voice>` (default `Samantha`,
365+
200 wpm). `stop()` kills via `killall say`.
364366
Platform-isolated for future Windows/Linux porting.
365367
- `mock` — 200 ms simulated delay. No audio output. For automated testing.
366368
- anything else / missing Azure keys — `tts:start` returns `{ok:false, message}`
@@ -545,6 +547,32 @@ Zoom / Google Meet / Microsoft Teams
545547
them into the central flow without rewriting business logic. Persistent
546548
recoverable state (missing device/config/permission) keeps its inline
547549
setup/panel UI, while transient failures surface as toasts.
550+
- **TTS voice selection (added 2026-09-03)**: the Settings → Voice section
551+
selects a voice via a searchable combobox that persists through `preferences`
552+
(`ttsVoiceId`) and is threaded into the existing TTS provider. Voice
553+
enumeration (`src/main/services/tts/voices.ts`) is dev-vs-prod gated on
554+
`!app.isPackaged`: packaged (production) builds expose a curated catalog of
555+
real Azure Neural voices only; dev (unpackaged) builds additionally enumerate
556+
macOS `say` system voices. The `SessionManager` receives the resolved voice
557+
through an injected resolver (`setTtsVoiceIdResolver`), so the session service
558+
stays free of `electron`/filesystem dependencies and remains unit-testable.
559+
**Known limitation**: macOS `say` exposes no gender metadata, so system voice
560+
gender is `unknown`. **Picker (2026-09-03)**: the picker (`VoicePicker.tsx`,
561+
Popover + Command/cmdk) lists **all** voices in two groups — "Azure voices"
562+
and "macOS system voices" (dev) — with type-to-search filtering by name and
563+
id; the earlier Female/Male gender Select and gender-filtered Voice Select
564+
were removed, and `ttsVoiceGender` was dropped from `AppPreferences`. This
565+
"one source of truth" rule keeps the production Azure TTS path and the "no
566+
parallel implementation" constraint intact — the Test Voice action reuses the
567+
same `TtsManager.onTranslationText()` pipeline. **Voice routing (2026-09-03
568+
fix)**: `createTtsProvider()` routes any non-Azure voice id (i.e. a macOS
569+
system voice, dev-only) to the `say` provider, so a system voice is never sent
570+
to the Azure SDK.
571+
**Test Voice isolation (2026-09-03 fix)**: the `tts:test` handler runs on its
572+
own `TtsManager` instance (not the shared session manager), starts the audio
573+
output it needs, and self-terminates after the test phrase — so a test never
574+
leaves TTS/audio active and never blocks a subsequent "/ Start Meeting" with
575+
"already running".
548576
- **Microphone capture in the renderer, permission in the main process**
549577
(see "Microphone pipeline" above): zero native dependencies, exact
550578
deviceId matching, and macOS TCC handled via `systemPreferences`.

docs/CHANGELOG.md

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,131 @@
33
Every agent working on this repository MUST append a dated entry describing
44
their changes after finishing work.
55

6+
## 2026-09-03 — Searchable Voice picker replaces the gender + voice selects
7+
8+
- **Removed the Voice gender (Female/Male) Select and the gender-filtered Voice
9+
Select** from the Settings → Voice section. `ttsVoiceGender` was dropped from
10+
`AppPreferences` (`packages/shared/index.ts`) and from
11+
`src/main/ipc/preferences.ts` (default, `loadPreferences`, and
12+
`preferences:set` all updated); the `selectedGender`/`onSelectVoiceGender`
13+
props were removed from `App.tsx`, `SettingsScreen.tsx`, and `TtsPanel.tsx`.
14+
- **Added a searchable voice picker** (`src/renderer/components/VoicePicker.tsx`,
15+
built on new shadcn `ui/popover.tsx` + `ui/command.tsx` primitives pulled from
16+
`@radix-ui/react-popover` and `cmdk`). It lists **all** voices in two groups —
17+
**Azure voices** (22) and **macOS system voices** (184, dev) — with
18+
type-to-search filtering by name **and** id. `TtsPanel.tsx` now renders just
19+
the Voice label + `VoicePicker` + Test Voice.
20+
- **Verified via CDP**: Settings → Voice shows one Voice combobox (no "Voice
21+
gender"); opening it shows Azure voices + macOS system voices; searching
22+
"prabhat" returns only `Prabhat (IN)` and "alice" matches the Alice system
23+
voice; selecting updates the trigger and persists `ttsVoiceId` (e.g.
24+
`en-IN-NeerjaNeural`). Test Voice and Start Meeting still behave correctly.
25+
- **Validation**: type-check clean; tests 80/80; build OK; ESLint 0 errors
26+
(removed the `cmdk-input-wrapper` attribute to satisfy `react/no-unknown-
27+
property`); Prettier clean.
28+
- Added deps: `cmdk@^1.1.1`, `@radix-ui/react-popover@^1.1.23`.
29+
30+
## 2026-09-03 — Make the Voice gender filter actually visible (Female/Male splits the list)
31+
32+
- `TtsPanel` no longer mixes macOS system voices (gender `unknown`) into the
33+
gender-filtered Azure list. **Female → only female Azure voices; Male → only
34+
male Azure voices.** The only macOS voice shown is a currently persisted
35+
system selection, kept so it is not lost.
36+
- Verified via CDP: with gender Male the Voice dropdown lists exactly Guy,
37+
Christopher, Eric, Roger, Steffan, Tony, Ryan, Thomas, William, Prabhat (IN),
38+
Liam (plus any retained macOS selection). Female uses the identical code
39+
branch.
40+
- Validation: type-check, tests 80/80, build OK, ESLint 0 errors, Prettier
41+
clean.
42+
43+
## 2026-09-03 — Fix: silent Test Voice + "TTS is already running" on Start Meeting
44+
45+
Follow-up bug fix to the voice-selection feature.
46+
47+
- **No audio on Test Voice**: `AudioOutputManager.writeAudio()` silently drops
48+
audio when the audio output manager is inactive, and `tts:test` never started
49+
it. The handler now starts `audioOutput` (if not active) before speaking the
50+
test phrase, so the selected voice is actually audible. Also, non-Azure/system
51+
voice ids route to the `say` provider (never Azure).
52+
- **"TTS is already running" on Start Meeting**: `tts:test` used the shared
53+
singleton `ttsManager` and left it active, so a later session start returned
54+
"TTS is already running." `tts:test` now uses an **independent `TtsManager`**
55+
that starts, speaks the test phrase, then self-terminates (`tts:spoken`
56+
`setImmediate` stop) and stops the audio output it started, emitting
57+
`tts:stopped` so the renderer returns to Off. The shared session manager is
58+
untouched by a test, so Start Meeting works without the error.
59+
- **Gender filter retained**: Filters the Azure voice list by documented Azure
60+
gender; dev macOS system voices are appended regardless. Verified via CDP:
61+
Settings → Voice → gender Male → select voice → Test Voice shows the text with
62+
provider "Say", badge Active, then auto-returns to Off; Start Meeting
63+
completes with no "TTS is already running" error.
64+
- **Validation**: type-check, tests 80/80, build OK, ESLint 0 errors, Prettier
65+
clean.
66+
67+
## 2026-09-03 — Fix: "Speech playback failed" when selecting/testing a voice
68+
69+
Follow-up bug fix to the voice-selection feature.
70+
71+
- **Root cause**: `createAzureTtsProvider` passed the selected voice id to the
72+
Azure SDK unconditionally. A macOS `say` voice id (dev-only, selectable only
73+
when a system voice is picked) is not an Azure voice — Azure rejected it →
74+
`tts:error` → "Speech playback failed". Selecting an Azure voice without
75+
`AZURE_SPEECH_KEY`/`AZURE_SPEECH_REGION` also threw immediately.
76+
- **Fix 1 — voice-aware provider routing**: `provider.ts`
77+
`createTtsProvider(voiceId?)` routes any non-Azure voice id to the `say`
78+
provider via new `voiceIsAzure()` (`voices.ts`), so a macOS system voice is
79+
always synthesized locally and never reaches the Azure SDK. Dev-only;
80+
production still exposes Azure voices only, so the production Azure path is
81+
unchanged.
82+
- **Fix 2 — system voices selectable in dev**: `TtsPanel` lists macOS system
83+
voices (marked "(macOS)", gender `unknown`) in the Voice dropdown in dev, so
84+
users can test tones fully offline with `say` and no Azure keys. Azure voices
85+
still need the Azure credential to synthesize.
86+
- **Tests**: 2 new `voiceIsAzure` tests; suite 80 passing (was 78).
87+
- **Validation**: type-check, tests 80/80, build OK, ESLint 0 errors, Prettier
88+
clean. CDP-verified: selecting a macOS voice + Test Voice completes with no
89+
error toast and no console errors.
90+
91+
## 2026-09-03 — TTS voice selection: gender + voice dropdowns, Test Voice, dev system voices
92+
93+
Voice selection feature for Settings → Voice (no commit/push).
94+
95+
- **Shared types** (`packages/shared/index.ts`): `VoiceGender`
96+
(`female | male | unknown`), `TtsVoiceSource` (`azure | system`), `TtsVoice`,
97+
`ListVoicesResult`; `AppPreferences` gains `ttsVoiceGender` (default
98+
`female`) and `ttsVoiceId` (default `null`); `ElectronAPI` gains
99+
`getTtsVoices()` and `testTtsVoice()`.
100+
- **Voice catalog/enumeration** (`src/main/services/tts/voices.ts`):
101+
curated `AZURE_VOICES` (22 real Azure Neural English voices with documented
102+
gender, default `en-US-JennyNeural`); `parseSayVoices()` for `say -v '\?'`
103+
(system voice gender is `unknown` — macOS exposes no gender metadata, so
104+
system voices are excluded from the Female/Male filter, documented
105+
limitation); `listVoices(development)` + `normalizeSelectedVoiceId()`
106+
(production restricted to curated Azure ids; dev passes through). macOS
107+
system voices exposed only when `!app.isPackaged` — Azure-only in packaged
108+
builds.
109+
- **Provider voice threading**: `createTtsProvider(voiceId?)` → Azure
110+
(`voiceId` beats `AZURE_TTS_VOICE`, fallback `en-US-JennyNeural`) and `say`
111+
(`-v <voice>`, default `Samantha`); `TtsManager.start(..., voiceId?)`.
112+
- **IPC** (`src/main/ipc/tts.ts`): `tts:start` resolves the persisted voice;
113+
new `tts:list-voices` and `tts:test` (stops active TTS, restarts with the
114+
selected voice, then feeds a fixed test sentence through the existing
115+
`ttsManager.onTranslationText(...)` pipeline — no parallel TTS path).
116+
`resolveTtsVoiceId()` feeds the SessionManager via an injected resolver
117+
(`setTtsVoiceIdResolver`), keeping the session service electron-free and
118+
unit-testable.
119+
- **Preferences** (`src/main/ipc/preferences.ts`): exports `loadPreferences`,
120+
persists `ttsVoiceGender`/`ttsVoiceId` so a selection survives restart.
121+
- **UI** (`TtsPanel.tsx`, `App.tsx`, `SettingsScreen.tsx`, new
122+
`useTtsVoices.ts`): Voice gender Select (Female/Male) + Voice Select filtered
123+
by gender + dev-only "system voices available in dev" label + **Test Voice**
124+
button.
125+
- **Tests** (`tests/voices.test.ts`): 10 new tests; suite 78 passing (was 68).
126+
- **Validation**: type-check clean, tests 78/78, build OK, ESLint 0 errors,
127+
Prettier clean; CDP-smoke at 480px (gender + voice dropdowns populate, male
128+
filter shows male Azure voices, `getTtsVoices` returns 206 voices in dev,
129+
no horizontal overflow, correct theme).
130+
6131
## [1.0.0] - 2026-08-28
7132

8133
First public open-source release of the Urdu → English Voice Interpreter for

docs/CURRENT_STATE.md

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,116 @@
22

33
_Last updated: 2026-09-03_
44

5+
## TTS voice selection: searchable VoicePicker, Test Voice, dev system voices
6+
7+
Completed (2026-09-03). Voice preference feature for the Settings → Voice section.
8+
9+
### What is done
10+
- **Voice model (shared)**`packages/shared/index.ts` adds `VoiceGender`
11+
(`female | male | unknown`), `TtsVoiceSource` (`azure | system`),
12+
`TtsVoice { id; name; gender; source }`, and `ListVoicesResult { ok; voices;
13+
development; message? }`. `AppPreferences` retains only
14+
`ttsVoiceId: string | null` (default `null`) — the gender preference was
15+
removed. `ElectronAPI` (preload bridge) gains `getTtsVoices()` and
16+
`testTtsVoice()`.
17+
- **Voice catalog + enumeration**`src/main/services/tts/voices.ts`
18+
- `AZURE_VOICES`: curated catalog of 22 real, documented Azure Neural English
19+
voices (en-US/en-GB/en-AU/en-IN/en-CA) with their documented gender. Safe
20+
default `en-US-JennyNeural` (female).
21+
- `parseSayVoices()`: parses `say -v '\?'` output. macOS `say` exposes no
22+
gender metadata, so **system voice gender is `unknown`** (documented
23+
limitation, not guessed). Handles multi-word + parenthesized names ("Bad
24+
News", "Eddy (English)") by locating the `xx_YY` locale token before `#`.
25+
- `listVoices(development)`: Azure always + macOS system voices in dev only.
26+
- `normalizeSelectedVoiceId(storedId, development)`: production restricts to
27+
the curated Azure ids (stale dev system ids never reach production); empty
28+
→ default.
29+
- **Dev-vs-prod gating** — macOS system voices exposed only when
30+
`!app.isPackaged` (dev/unpackaged). Production (packaged) lists Azure only.
31+
Detection lives in the IPC layer (`isTtsDevelopment()`), keeping `voices.ts`
32+
pure and unit-testable (no `require('electron')`).
33+
- **Provider voice threading**`createTtsProvider(voiceId?)` flows the
34+
selected id into Azure (`createAzureTtsProvider` — voiceId beats
35+
`AZURE_TTS_VOICE`, fallback `en-US-JennyNeural`) and `say`
36+
(`createSayTtsProvider``-v <voice>`, default `Samantha`). `TtsManager.start`
37+
accepts `voiceId`.
38+
- **IPC**`src/main/ipc/tts.ts`: `tts:start` resolves the persisted voice;
39+
`tts:list-voices` returns `ListVoicesResult`; `tts:test` stops active TTS,
40+
restarts with the selected voice, and feeds a fixed test sentence through the
41+
existing TTS pipeline (`ttsManager.onTranslationText(...)`) — no parallel TTS
42+
path. `resolveTtsVoiceId()` exported; the SessionManager is fed it via an
43+
injected resolver (`setTtsVoiceIdResolver`) so the session service stays
44+
electron-free and unit-testable.
45+
- **Preferences persistence**`src/main/ipc/preferences.ts` (exports
46+
`loadPreferences`) persists `ttsVoiceId`; `preferences:set` persists it so a
47+
selection survives restart. (`ttsVoiceGender` removed.)
48+
- **UI — searchable VoicePicker**`TtsPanel.tsx` Voice section now uses a
49+
single searchable combobox (`VoicePicker.tsx`, built on `Popover` +
50+
`Command`/cmdk) that lists **every** available voice — the **Voice gender**
51+
Select and the gender-filtered **Voice** Select were both removed. The picker
52+
groups voices into **"Azure voices"** (22) and, in dev, **"macOS system
53+
voices"** (184); typing filters by name **and** id. A dev-only "system voices
54+
available in dev" label shows when `development`, plus a **Test Voice** button
55+
that calls `window.electron.testTtsVoice()`. Wired through `App.tsx`
56+
`SettingsScreen.tsx` via new `useTtsVoices()` hook (loads on mount) and the
57+
existing `usePreferences()`.
58+
- **Tests**`tests/voices.test.ts`: 12 tests for `parseSayVoices` (simple,
59+
multi-word/parenthesized, missing locale, blank lines, gender `unknown`),
60+
catalog validity, default voice, `normalizeSelectedVoiceId`
61+
(dev passthrough / empty fallback / prod restriction), and `voiceIsAzure`.
62+
Project suite: **80 tests pass**.
63+
- **Validation**`npm run type-check` clean; `npm test` 80/80; `npm run build`
64+
OK (`dist/renderer/index.html` present); ESLint 0 errors; Prettier clean.
65+
CDP-smoke-tested against the running app at min width: Settings → Voice shows
66+
one searchable Voice combobox, `getTtsVoices` returns **206** voices (22 Azure
67+
+ ~184 macOS system) with `development:true`, searching "prabhat" returns only
68+
`Prabhat (IN)`, Test Voice + Start TTS buttons present, no horizontal overflow
69+
(`scrollW == winW`), theme correct (dark).
70+
71+
### What remains
72+
- None for this feature. Requires a real Azure credential to hear the Test Voice /
73+
Meeting TTS with a selected Azure voice (provider path reused; already covered
74+
by the existing TTS pipeline).
75+
- Manual tasting of a specific macOS `say` voice in dev (optional, via system
76+
voice list).
77+
78+
### Bug fix — "Speech playback failed" / "TTS is already running" when selecting/testing voices (2026-09-03)
79+
- **Root cause (A) — silent "Speech playback failed"**: `createAzureTtsProvider`
80+
passes the selected voice id to the Azure SDK. (a) A macOS `say` voice id
81+
(dev-only; never an Azure voice) sent to Azure is rejected by the SDK →
82+
`tts:error`. (b) `AudioOutputManager.writeAudio()` **silently drops audio when
83+
the audio output manager is inactive** (`if (!this.active || !this.provider)
84+
return`), and the `tts:test` handler never started audio output — so the test
85+
synthesizes text and shows it but nothing is audible.
86+
- **Root cause (B) — "TTS is already running"**: `tts:test` used the **shared
87+
singleton `ttsManager`** and left it **active**. A subsequent `/ Start
88+
Meeting` called session → `ttsManager.start()` again → `{ok:false,
89+
message:'TTS is already running.'}` → session aborted at the TTS stage.
90+
- **Fix A (routing + audio)**`provider.ts` routes any non-Azure voice id to
91+
the `say` provider (`voiceIsAzure()` in `voices.ts`), and the `tts:test`
92+
handler now starts `audioOutput` (if not already active) before feeding the
93+
test phrase so the audio is actually played.
94+
- **Fix B (independent test manager)**`tts:test` now creates its **own
95+
`TtsManager`** instance (same provider path, no parallel TTS implementation)
96+
that starts, speaks the test phrase, then **self-terminates** (`setImmediate`
97+
on `tts:spoken``stop()`) and stops the audio output it started, emitting
98+
`tts:stopped` so the renderer returns to idle. The shared meeting/session
99+
`ttsManager` is never touched by a test, so starting a meeting no longer hits
100+
"already running".
101+
- **Gender filter removed (2026-09-03)**: the Female/Male dropdown and the
102+
gender-filtered Voice dropdown were both replaced by a single searchable
103+
**VoicePicker** showing all voices (Azure + macOS in dev). No gender UI or
104+
`ttsVoiceGender` preference remains.
105+
- **Verified via CDP (TTS_PROVIDER=say)**: Settings → Voice now shows a single
106+
Voice combobox (no "Voice gender"); opening it lists two groups — **Azure
107+
voices** (22) and **macOS system voices** (184); searching "prabhat" yields
108+
only `Prabhat (IN)`, searching "alice" matches the Alice system voice; picking
109+
a voice updates the trigger and persists `ttsVoiceId` (e.g.
110+
`en-IN-NeerjaNeural`). **Test Voice** shows "Hello, this is a test…", provider
111+
"macOS Say", badge Active, then auto-returns to Off; Start Meeting completes
112+
with no "TTS is already running" error. Suite **80 passing**;
113+
type-check/build/eslint/format clean.
114+
5115
## UX/UI infrastructure: Settings layout overflow fix + centralized error handling & toasts
6116

7117
Completed (2026-09-03). Two-part task, both verified end-to-end.

0 commit comments

Comments
 (0)