Skip to content

Commit ab039be

Browse files
feat: ElevenLabs-styled MicSelector + provider-aware TTS voice listing + audio quality fix
- Replace MicrophonePanel/AudioLevelMeter with ElevenLabs DevLab-styled MicSelector (LiveWaveform + device dropdown + record/pause/play/trash) - Fix mic sound-check playback: ScriptProcessorNode captures sequential non-overlapping PCM (replaces broken AnalyserNode polling that caused distorted audio from overlapping reads) - Provider-aware TTS voice listing: voices match active TTS_PROVIDER (azure/say/mock), resolveTtsVoiceId rejects invalid voice ids - Remove dead UI: Start TTS, Speech & Translation section, Test My Microphone, level prop on MicSelector - 88/88 tests, type-check, lint, build clean
1 parent 811dd59 commit ab039be

15 files changed

Lines changed: 845 additions & 593 deletions

File tree

docs/ARCHITECTURE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ Electron
4545
├── services/useTranslation.ts (translation events → english text)
4646
├── services/useTts.ts (TTS state)
4747
├── services/useAudioOutput.ts (WebAudio playback, device selection)
48-
├── components/ (MicrophonePanel, AudioLevelMeter, SttPanel)
48+
├── components/ (MicSelector, AudioOutputPanel, VoicePicker, TtsPanel, PipelinePanel, SetupPanel, AudioLevelMeter)
4949
├── pages/ (HomeScreen; LiveTranslationScreen = subtitle stub)
5050
└── styles/ (App.css)
5151

docs/CHANGELOG.md

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,104 @@
33
Every agent working on this repository MUST append a dated entry describing
44
their changes after finishing work.
55

6+
## 2026-09-03 — TTS voice listing provider-aware + Settings cleanup (Start TTS, Speech & Translation, mic-selector, Test My Microphone)
7+
8+
- **TTS voice listing is now provider-aware.** `listVoices(development, provider)`
9+
in `src/main/services/tts/voices.ts` returns the catalog matching the active
10+
synthesizer: `TTS_PROVIDER=say` → only macOS/system voices (Azure ids are
11+
never listed because `say` cannot speak them); `TTS_PROVIDER=azure` → Azure
12+
voices always + macOS system voices in dev; `mock` → Azure only; unknown →
13+
none. New pure helper `resolveTtsProviderName()` reads `TTS_PROVIDER` from the
14+
env (default `mock`) and is unit-tested.
15+
- **`tts:list-voices` IPC threads the resolved provider** into `listVoices` and
16+
returns it in `ListVoicesResult.provider` (`packages/shared/index.ts`),
17+
so the catalog the UI shows always matches the runtime synthesizer.
18+
- **`resolveTtsVoiceId()` is provider-aware too**: with `TTS_PROVIDER=say` a
19+
persisted Azure id is rejected (falls back to the `say` provider default)
20+
because `say` cannot synthesize it; azure/mock keeps the existing
21+
`normalizeSelectedVoiceId()` behavior. This prevents listing an Azure voice
22+
under `say` or trying to speak an Azure voice with `say`.
23+
- **Removed the redundant "Start TTS" / "Stop TTS" button** in Settings → Voice.
24+
Audited: it only enabled `translationActive`, was duplicated by the
25+
SessionManager during a meeting, and overlapped with the existing **Test Voice**
26+
button (independent, self-terminating `tts:test`). `TtsPanel` now keeps voice
27+
selection + Test Voice + status/current-text display. `handleTtsStart` and the
28+
`onTtsStart`/`onTtsStop` props were removed end-to-end.
29+
- **Removed the "Speech & Translation" Settings section** and its dead manual
30+
control surface: `SttPanel.tsx` and `TranslationPanel.tsx` were deleted; the
31+
`speech` nav entry, and the `SttPanel`/`TranslationPanel` renders and their
32+
props were removed from `SettingsScreen.tsx`. Dead `handleSttStart` /
33+
`handleSttStop` and the `onSttStart`/`onSttStop`/`onTranslationStart`/
34+
`onTranslationStop` props were removed from `App.tsx`. The shared `stt`/
35+
`translation` hooks and the meeting pipeline are untouched (STT/translation
36+
still run via `/ Start Meeting`).
37+
- **Fixed `MicSelector` sound-check playback** ("couldn't hear recorded sound"):
38+
the `<audio>` element (`HTMLAudioElement`) cannot decode **any** audio format
39+
in this AVMedia Chromium build (conclusive: WAV, webm, and via
40+
`createMediaElementSource` all yield `networkState=3` /
41+
`MEDIA_ERR_SRC_NOT_SUPPORTED`). Rewrote the capture and playback to use the
42+
repo's own raw-PCM path: the mic stream is tapped with a second
43+
`AudioContext` + `ScriptProcessorNode` at 24 kHz, accumulating sequential,
44+
non-overlapping PCM frames into a Float32 array via event-driven capture.
45+
Playback uses `AudioContext.createBuffer(source).connect(destination).start()`
46+
(the same pattern `useAudioOutput` uses, proven audible in this build). Pause/
47+
resume tracks elapsed frames via a poll timer and re-creates a sub-buffer from
48+
the paused offset on resume. Duplicate `handleStreamEnd` calls (LiveWaveform
49+
cleanup) are guarded by a `scriptProcessorRef !== null` check.
50+
`src/renderer/components/MicSelector.tsx`.
51+
52+
- **Fixed `MicSelector` sound-check audio quality** (distorted/laggy playback):
53+
the previous implementation used `AnalyserNode.getFloatTimeDomainData()` polled
54+
via `setInterval(30ms)` to capture PCM. This is fundamentally broken for
55+
sequential capture: `getFloatTimeDomainData` returns a sliding window of the
56+
last `fftSize` samples, not sequential non-overlapping chunks. At 24 kHz with
57+
`fftSize=1024`, the analyser refreshes every ~42 ms, but 30 ms polls land
58+
between frame boundaries — causing overlapping reads (~300 duplicate samples per
59+
read), which produces pitch-shifted, distorted playback. CDP-verified:
60+
`getFloatTimeDomainData` at 5 ms apart returns 100% identical data (analyser
61+
hasn't refreshed), at 42 ms apart returns 0% overlap. Replaced with
62+
`ScriptProcessorNode` (`createScriptProcessor(4096, 1, 1)`) which fires
63+
`onaudioprocess` once per buffer with guaranteed sequential, non-overlapping
64+
`Float32Array` chunks. CDP timing: 3 s recording → ~2.79 s playback (delta is
65+
ScriptProcessorNode startup + React teardown latency, not sample-rate error).
66+
`src/renderer/components/MicSelector.tsx`.
67+
68+
- **Fixed the `MicSelector` sound-check Play → Pause flow.** The `LiveWaveform`
69+
was rendered with `key={checkState}`, which remounted the component on every
70+
state change (`idle → recording → recorded → playing`). Each remount
71+
re-ran the microphone setup/teardown effect — tearing the stream down,
72+
re-fetching the mic, and churning `audio.onended`/playback state so the
73+
`playing` state wouldn't hold and the Play→Pause transition failed.
74+
Removed the `key` so a single stable `LiveWaveform` instance persists; it now
75+
clears its own history on teardown (`active=false`), and recording
76+
stop/`onStreamEnd` sealing works once. `src/renderer/components/MicSelector.tsx`.
77+
78+
- **Replaced the microphone UI with an ElevenLabs-styled `MicSelector`**
79+
(`src/renderer/components/MicSelector.tsx`). The ElevenLabs registry is
80+
rate-limited (HTTP 429 — `npx @elevenlabs/cli@latest components add
81+
mic-selector` and the `shadcn` fallback both fail), so — consistent with how
82+
`VoicePicker` and the vendored `LiveWaveform` were handled — a local component
83+
mirrors the ElevenLabs DevLab `mic-selector` **sound-check card**: a live
84+
`LiveWaveform` (barWidth 3, scrolling, fade edges), a microphone device
85+
dropdown (popover with mic glyph + check), and a record / pause / play / trash
86+
control row plus a mute toggle. Record/play is a device test only: the
87+
`LiveWaveform` opens ONE stream and hands it to a `MediaRecorder` via
88+
`onStreamReady`, so capture is never duplicated with the meeting pipeline
89+
(which still owns real recording). Wired through the existing `useMicrophone`
90+
device state (`micDevices`/`selectedDeviceId`/`permission`/`error`/
91+
`onSelectDevice`); the `level` prop and `AudioLevelMeter` were dropped.
92+
Old `MicrophonePanel.tsx` and its now-unconsumed `onMicStart`/`onMicStop`
93+
props were deleted.
94+
- **Removed the "Test My Microphone" diagnostics button** (dead — it re-ran
95+
`microphone.start`, duplicating the meeting path) and its `onDeviceTest`
96+
prop. Diagnostics now lists mic / TTS / audio-output / current-stage status
97+
(STT and Translation rows removed with the Speech & Translation section).
98+
- **Validation**: `npm run type-check` clean; `npm test` **88/88** (5 new:
99+
`resolveTtsProviderName`, per-provider `listVoices` for azure/mock/say/
100+
unknown); `npm run build` OK (`dist/renderer/index.html` present); `npm run
101+
format:check` clean; `npm run lint` **0 errors** (13 pre-existing warnings
102+
unchanged, incl. hook/`no-unused-vars` in App/useSetup/tests).
103+
6104
## 2026-09-03 — Voice picker restyled to ElevenLabs UI + search by gender/country/source
7105

8106
- **Rewrote `VoicePicker.tsx` to mirror the ElevenLabs UI `voice-picker`

docs/CURRENT_STATE.md

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,85 @@
22

33
_Last updated: 2026-09-03_
44

5+
## TTS voice listing provider-aware + Settings cleanup
6+
7+
Completed (2026-09-03). Multi-part refresh of the Settings surface + TTS voice
8+
catalog so they match the active runtime provider.
9+
10+
### What is done
11+
- **Provider-aware voice listing.** `listVoices(development, provider)` in
12+
`src/main/services/tts/voices.ts` now returns the catalog for the active
13+
synthesizer instead of always listing Azure (+system in dev):
14+
- `TTS_PROVIDER=say`**only macOS system voices** (never Azure ids — `say`
15+
cannot speak them).
16+
- `TTS_PROVIDER=azure` → Azure voices always + macOS system voices in dev.
17+
- `mock` → Azure only; unknown provider → no voices.
18+
- New pure, unit-tested `resolveTtsProviderName()` reads `TTS_PROVIDER`.
19+
- **`tts:list-voices` threads the provider and returns it** via
20+
`ListVoicesResult.provider` (`packages/shared/index.ts`), so the picker list
21+
always matches the runtime synthesizer.
22+
- **`resolveTtsVoiceId()` is provider-aware**: under `TTS_PROVIDER=say` a
23+
persisted Azure id is dropped (falls back to the `say` provider default)
24+
because `say` cannot synthesize it; azure/mock keeps the existing
25+
`normalizeSelectedVoiceId()` behavior.
26+
- **"Start TTS" removed** (audited redundant: gated on `translationActive`,
27+
duplicated by SessionManager in-meeting, overlapped by the self-terminating
28+
Test Voice). `TtsPanel` keeps voice selection + Test Voice + status.
29+
- **"Speech & Translation" Settings section removed** as dead manual control;
30+
`SttPanel.tsx`/`TranslationPanel.tsx` deleted; dead `handleSttStart`/
31+
`handleSttStop` and `onStt*`/`onTranslation*` props removed. STT/translation
32+
still run via `/ Start Meeting` (hooks + pipeline untouched).
33+
- **Fix applied**: removed `key={checkState}` from the `LiveWaveform` inside
34+
`MicSelector` — it was remounting the waveform on every state change, tearing
35+
the recording stream down and making the Play → Pause sound-check flow fail.
36+
A single persistent waveform instance now handles record/stop/play cleanly.
37+
38+
- **Fix applied**: `MicSelector` sound-check playback reworked to use the
39+
repo's own raw-PCM + `createBufferSource` path. The `<audio>` element
40+
(`HTMLAudioElement`) cannot decode **any** audio format in this AVMedia
41+
Chromium build (conclusive: WAV, webm, via `createMediaElementSource` all
42+
yield `networkState=3` / MEDIA_ERR_SRC_NOT_SUPPORTED). The implementation:
43+
- **Capture**: taps the `LiveWaveform`'s single mic stream with a second
44+
`AudioContext` + `ScriptProcessorNode` (`createScriptProcessor(4096, 1, 1)`)
45+
at 24 kHz, capturing sequential, non-overlapping PCM frames via
46+
`onaudioprocess` events into a Float32 array. (Replaced the previous
47+
`AnalyserNode` + `setInterval` approach, which produced overlapping reads
48+
and distorted playback — CDP-verified: `getFloatTimeDomainData` returns a
49+
sliding window, not sequential chunks.)
50+
- **Play**: `AudioContext.createBuffer(source).connect(ctx.destination).start()`
51+
(the repo's own `useAudioOutput` pattern, proven audible in this build).
52+
- **Pause/Resume**: a poll timer tracks elapsed frames; pause stores the
53+
current offset and closes the source; play creates a new sub-buffer from the
54+
offset onward. `onended` restores the recorded state after completion.
55+
- Duplicate `handleStreamEnd` calls from LiveWaveform cleanup are guarded by a
56+
`scriptProcessorRef !== null` check.
57+
Combined with the `key={checkState}` removal, the full record → stop → play →
58+
pause → play-again → finish → trash flow works. `src/renderer/components/
59+
MicSelector.tsx`.
60+
61+
- **ElevenLabs-styled `MicSelector`** replaces the microphone UI
62+
(`MicrophonePanel.tsx`, `AudioLevelMeter.tsx` deleted). The ElevenLabs
63+
registry is rate-limited (HTTP 429), so — consistent with the vendored
64+
`LiveWaveform` and the custom `VoicePicker` — a local component mirrors the
65+
ElevenLabs DevLab `mic-selector` **sound-check card**: a live `LiveWaveform`,
66+
a microphone device dropdown, and record / pause / play / trash + mute
67+
controls. It is a device-test surface only: `LiveWaveform` opens ONE stream
68+
handed to the PCM recorder via `onStreamReady` — capture is never duplicated
69+
with the meeting pipeline (which still owns real recording). Drives device
70+
selection through the existing `useMicrophone` state; the `level` prop and
71+
`AudioLevelMeter` were dropped.
72+
- **"Test My Microphone" diagnostics button removed** (dead: duplicated
73+
`microphone.start`). Diagnostics shows mic / TTS / audio-output / stage.
74+
75+
### Validation
76+
- `npm run type-check` clean; `npm test` **88/88** (5 new provider/voice tests);
77+
`npm run build` OK (`dist/renderer/index.html` present); `npm run format:check`
78+
clean; `npm run lint` 0 errors (13 pre-existing warnings unchanged).
79+
80+
### What remains / next
81+
- None for this cleanup. The TTS provider/voice path is unchanged for `azure`;
82+
`say` (currently active in `.env`) now lists only macOS system voices.
83+
584
## TTS voice selection: searchable VoicePicker, Test Voice, dev system voices
685

786
Completed (2026-09-03). Voice preference feature for the Settings → Voice section.

packages/shared/index.ts

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,11 @@ export interface ListVoicesResult {
122122
voices: TtsVoice[];
123123
/** True when the app is running unpackaged (macOS system voices available). */
124124
development: boolean;
125+
/**
126+
* The active TTS provider that produced this catalog — `say` lists only
127+
* macOS system voices; `azure`/`mock` list Azure voices (system in dev).
128+
*/
129+
provider?: 'azure' | 'say' | 'mock' | 'none';
125130
message?: string;
126131
}
127132

src/main/ipc/tts.ts

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,12 @@
1919
import { ipcMain, BrowserWindow, app } from 'electron';
2020
import { TtsManager } from '../services/tts/manager';
2121
import type { AudioOutputManager } from '../services/audio-output/manager';
22-
import { listVoices, normalizeSelectedVoiceId } from '../services/tts/voices';
22+
import {
23+
listVoices,
24+
normalizeSelectedVoiceId,
25+
resolveTtsProviderName,
26+
voiceIsAzure,
27+
} from '../services/tts/voices';
2328
import { loadPreferences } from './preferences';
2429

2530
const TEST_TEXT = 'Hello, this is a test of the selected voice.';
@@ -34,10 +39,24 @@ export function isTtsDevelopment(): boolean {
3439
return !app.isPackaged;
3540
}
3641

37-
/** The provider id currently selected by the user, resolved for this environment. */
42+
/**
43+
* The voice id currently selected by the user, resolved for this environment
44+
* AND the active TTS provider. A persisted id that the active provider cannot
45+
* synthesize is dropped so the provider falls back to its own default voice:
46+
* - `say`: only macOS system voices are usable (an Azure id is rejected).
47+
* - azure/mock: only curated Azure ids in production (system voices in dev).
48+
*/
3849
export function resolveTtsVoiceId(): string | null {
3950
const preferences = loadPreferences();
40-
return normalizeSelectedVoiceId(preferences.ttsVoiceId, isTtsDevelopment());
51+
const stored = preferences.ttsVoiceId?.trim();
52+
const provider = resolveTtsProviderName();
53+
54+
if (provider === 'say') {
55+
if (!stored || voiceIsAzure(stored)) return null;
56+
return stored;
57+
}
58+
59+
return normalizeSelectedVoiceId(stored, isTtsDevelopment());
4160
}
4261

4362
export function registerTtsIpc(getWindow: () => BrowserWindow | null, audioOutput: AudioOutputManager): void {
@@ -57,8 +76,13 @@ export function registerTtsIpc(getWindow: () => BrowserWindow | null, audioOutpu
5776

5877
ipcMain.handle('tts:list-voices', async (): Promise<import('@shared/index').ListVoicesResult> => {
5978
try {
60-
const { voices, development } = await listVoices(isTtsDevelopment());
61-
return { ok: true, voices, development };
79+
const provider = resolveTtsProviderName();
80+
const {
81+
voices,
82+
development,
83+
provider: resolvedProvider,
84+
} = await listVoices(isTtsDevelopment(), provider);
85+
return { ok: true, voices, development, provider: resolvedProvider };
6286
} catch (err) {
6387
const msg = err instanceof Error ? err.message : String(err);
6488
return { ok: false, voices: [], development: isTtsDevelopment(), message: msg };

src/main/services/tts/voices.ts

Lines changed: 67 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -115,28 +115,83 @@ function runSayList(): Promise<string> {
115115
});
116116
}
117117

118-
/** List all voices for the current environment. */
118+
/** TTS providers that can map a selected voice onto a playable synthesizer. */
119+
export type TtsProviderName = 'azure' | 'say' | 'mock' | 'none';
120+
121+
/**
122+
* Resolve the configured TTS provider name for the current process. Mirrors the
123+
* source of truth used by `createTtsProvider` so the voice catalog stays in
124+
* lock-step with the synthesizer actually used at runtime.
125+
*/
126+
export function resolveTtsProviderName(
127+
env: Record<string, string | undefined> = process.env,
128+
): TtsProviderName {
129+
const name = (env.TTS_PROVIDER || 'mock').toLowerCase();
130+
return name === 'azure' || name === 'say' || name === 'mock' ? name : 'none';
131+
}
132+
133+
/**
134+
* List all voices for the current environment, filtered to the active TTS
135+
* provider. Because a picked voice must always be audible with the synthesizer
136+
* that runs in this process:
137+
* - `say`: only macOS system voices (no Azure ids — they are not `say` voices).
138+
* - `azure`: Azure voices always, plus macOS system voices in dev only.
139+
* - `mock`: Azure voices only (mock produces no real audio; used in dev/tests).
140+
* - `none`: no voices (unknown provider).
141+
*/
119142
export async function listVoices(
120143
development: boolean,
121-
): Promise<{ voices: TtsVoice[]; development: boolean }> {
122-
const voices: TtsVoice[] = AZURE_VOICES.map((v) => ({
123-
id: v.id,
124-
name: v.name,
125-
gender: v.gender,
126-
source: 'azure',
127-
country: countryFromAzureId(v.id),
128-
}));
144+
provider: TtsProviderName,
145+
): Promise<{ voices: TtsVoice[]; development: boolean; provider: TtsProviderName }> {
146+
const voices: TtsVoice[] = [];
129147

130-
if (development) {
148+
if (provider === 'say') {
149+
// macOS system voices are the only voices the `say` synthesizer can use.
131150
try {
132151
const system = await runSayList();
133152
voices.push(...parseSayVoices(system));
134153
} catch {
135-
// macOS `say` enumeration failed — Azures voices still enumerated.
154+
// macOS `say` enumeration failed — return an empty system list.
136155
}
156+
return { voices, development, provider };
157+
}
158+
159+
if (provider === 'azure') {
160+
voices.push(
161+
...AZURE_VOICES.map((v) => ({
162+
id: v.id,
163+
name: v.name,
164+
gender: v.gender,
165+
source: 'azure' as const,
166+
country: countryFromAzureId(v.id),
167+
})),
168+
);
169+
if (development) {
170+
try {
171+
const system = await runSayList();
172+
voices.push(...parseSayVoices(system));
173+
} catch {
174+
// macOS `say` enumeration failed — Azure voices still enumerated.
175+
}
176+
}
177+
return { voices, development, provider };
178+
}
179+
180+
// mock: Azure voices only (mock is a dev/testing synthesizer with no audio).
181+
if (provider === 'mock') {
182+
voices.push(
183+
...AZURE_VOICES.map((v) => ({
184+
id: v.id,
185+
name: v.name,
186+
gender: v.gender,
187+
source: 'azure' as const,
188+
country: countryFromAzureId(v.id),
189+
})),
190+
);
191+
return { voices, development, provider };
137192
}
138193

139-
return { voices, development };
194+
return { voices, development, provider };
140195
}
141196

142197
/**

0 commit comments

Comments
 (0)