fix(msteams): a name the resolver cannot select is recovered structurally — witnessed stale live (#853, #797)#896
Draft
DmitriyG228 wants to merge 59 commits into
Draft
fix(msteams): a name the resolver cannot select is recovered structurally — witnessed stale live (#853, #797)#896DmitriyG228 wants to merge 59 commits into
DmitriyG228 wants to merge 59 commits into
Conversation
added 30 commits
July 20, 2026 00:03
…aker attribution The gmeet replay harness proves segmentation off per-channel, glow-named frames. The lane the attribution bugs actually live in (#797/#499/#539) is the mixed one: ONE audio stream for everybody, named only from out-of-band hints — and it had no offline oracle at all, so 'audio survived, attribution collapsed to seg_N' could only ever be observed live. replay-mixed.test.ts drives the REAL @vexa/mixed-pipeline from a stored session, replacing only the two non-deterministic dependencies (STT → a clock-keyed mock; the cut → an injected BoundarySource, since production downloads a pyannote model) and asserting who-spoke: deterministic across two runs, every name traceable to a recorded hint, no segment left anonymous, both ground-truth speakers present across the turn change. Wired into the package's replay script so gate:replay covers it. The golden is harvested from a real jitsi meeting (two scripted speakers with ground-truth WAVs) and distilled to the ~18s Anna->Boris window: 58 frames + 9 hints, real ts/hint timing with a synthetic speech-level waveform — replaying the real-audio cut and the synthetic one yields identical output (Anna:8.5s, Boris:6.3s), which is what lets the fixture ship at 27 KB gzipped instead of 1.2 MB. Two fixture properties are load-bearing and documented in the fixture README, both found the hard way: amplitude below the capture energy gate replays to ZERO segments, and a too-tight leading cut makes the short-UI-switch guard correctly reject the opening name as a tile flip (seg_0 — a fixture artifact, not a defect). The attribution checks are guarded on a non-empty replay so a silent fixture fails loud instead of greening three rows vacuously.
…ment that measures something real Two things. 1) RE-LAND the mixed-lane attribution oracle. It was reviewed on #831 but the squash-merge (2662ce2) captured the PR BEFORE the commit landed, so replay-mixed.test.ts and its golden are absent from main. Cherry- picked verbatim from 2eda66b and re-verified green here. It now sits in both the package test chain and the replay script, so gate:replay covers the lane the attribution bugs actually live in. 2) A latency instrument, and an honest account of why the obvious one is worthless. Wiring a lag oracle into the BATCH replay produced p50=3900ms/max=7800ms - pure artifact: those are exactly (session end - segment end), because the harness feeds every frame in a tight loop and the pipeline's submit/confirm cadence is WALL-CLOCK (speaker-streams.ts setInterval/Date.now), so nothing can confirm until all the audio is already in. That number is not reported and not gated; what the batch harnesses now assert instead is the invariant that IS meaningful there - nothing may be confirmed before its own audio exists. The real measurement needs the session fed at the rate it was spoken, so replay-paced.test.ts does that (opt-in, ~10s for the golden, wall-clock and therefore machine-sensitive, gating nothing unless LATENCY_BUDGET_P95_MS is set). Its causality check earned its keep immediately: the first run read -145ms because frames were paced to their START timestamp, while a capture frame is delivered when its audio ENDS. The number it now reports (~55ms p50 on the golden) is deliberately NOT presented as our latency: it reads below the 250ms mock STT round-trip, which is not physically coherent for audio that must be transcribed before it is confirmed, so segment end evidently tracks the confirmed prefix's audio timeline rather than the last chunk sent to STT. That semantic needs pinning before any budget is declared - documented in the harness header rather than papered over.
…nd an honest upper bound for the mixed lane Pins the semantic that blocked a latency budget. speaker-streams computes a segment's end as windowStartMs + totalSamples/sampleRate — the buffer's FULL extent at confirm time, so it tracks the newest audio fed rather than the span the text covers. Latency measured against it collapses to a constant (~55ms) however slow STT is, which is what the first version of this harness reported. The metric is now taken against the segment START: turn start -> confirmed text, the delay a user actually experiences. Validated as an instrument, not asserted: mock STT 100ms -> p50 503ms, 250ms -> 856ms, 600ms -> 1509ms. That is about 2xSTT + 300ms, because LocalAgreement confirmThreshold=2 needs TWO submissions (two round-trips) before a prefix confirms — so the confirm threshold, not buffering, is the dominant latency lever. A coherence check now fails the run if time-to-text ever drops below the STT round-trip it must pay, so the metric cannot silently degenerate into a moving target again. gmeet is gated at p95 <= 1500ms (observed 856ms, headroom for a regression guard rather than an aspiration). The mixed lane (zoom/teams/jitsi) is measured but deliberately NOT gated. Two things separate it from gmeet: it confirms per TURN, so confirmed-text timing is bounded by how long a speaker holds the floor (6.8-11.3s here) rather than by latency — hence first visible DRAFT is this lane's comparable number; and this harness injects a cut source that fires only on speaker change, whereas production's PyannoteSegmenter cuts continuously on speech/silence, so the pipeline is starved of the cuts that trigger early drafts (Anna's first turn drafts not at all until it ends). The numbers are therefore an upper bound produced by an unrepresentative double. Gating on that would be a green that proves nothing; a real budget needs a boundary source with production cut density, which recorded pyannote boundaries in captured-signal.v1 could supply.
…can be measured instead of approximated
The mixed lane (zoom/teams/jitsi) could not be gated on anything, because
its chunking is decided entirely by PyannoteSegmenter and a replay cannot
download the model. Every offline harness therefore substituted its own
cut source, and a number measured against a substitute is a number about
the substitute: the previous run reported 2.9-11.3s time-to-text and
Anna's opening turn drafting not at all, which was the harness starving
the pipeline of cuts rather than the pipeline being slow.
captured-signal.v1 gains a type-tagged BoundaryEvent, and the bot tees
production's OWN cuts into the recorded session by wrapping the real
segmenter through the existing makeSegmenter seam (so production keeps
its own cut source and the recorder only observes it, kind + tMs +
confidence, on the same epoch clock as frames and hints). Additive:
recorders and replays that predate it still work, and both harnesses say
out loud which chunking they are using.
Live-harvested on the lab: 216 frames, 76 hints, 27 real pyannote cuts.
With production's chunking the mixed lane now measures rather than
approximates:
attribution - both speakers named across the turn change, deterministic
latency - time-to-first-text p50 2780ms / max 4406ms
(vs gmeet 856ms, same paced clock, same 250ms mock STT)
That ~3x gap is real and is a product question, not a test threshold: the
mixed lane waits for a pyannote cut, which needs a PAUSE, where gmeet
submits per speaker every 100ms. Budgets are therefore per-lane
regression guards with headroom (gmeet 1500ms, mixed 6000ms), and the
harness prints the comparison so the gap stays visible rather than
normalized.
Two fixture artifacts found and excluded along the way, both of which
mimic the #797 signature: a window starting mid-turn makes the pipeline
cold-start and emit seg_0 (the full session attributes fine), and a
too-short leading turn trips the short-UI-switch guard. The golden is
therefore cut from the session's natural start; both traps are in the
fixture README.
…of completing silently The faults were never lost for lack of typing: @vexa/transcribe-whisper classifies every one (payment_required/unauthorized/unavailable/timeout, with the backend's own detail), and both pipelines hand them to onError. What was missing was the last hop — the composition root logged them to a console that dies with the container. So a meeting whose STT backend refused every chunk reached 'completed' indistinguishable from a silent room, which is why empty-transcript reports could not be diagnosed after the fact (docs/test-scenarios/terminal-seams.md carried this as an explicit GAP row). The bot now accumulates STT faults per kind and reports once, on the TERMINAL lifecycle event — the last thing anyone hears from a bot, and the only emit that is never an idempotent no-op at the receiver. A dead backend faults on every chunk (18 in a 2-minute live run), so counting and reporting once is both the honest summary and the only shape that cannot flood the control plane; the first fault of each kind still logs loudly for anyone tailing the container. Rides lifecycle.v1 as an additive field exactly as infra_fault does (additionalProperties: true — no schema or seal change). meeting-api ingests it into meeting.data.stt_fault, so it reaches the DB and the existing meeting.status_change webhook without a new event type. Deliberately NOT changed: the no_op gate on same-status re-emits (its idempotency is load-bearing), and the terminal's live banner — #552's UX half still names the transport layer, because the SSE the banner reads is fed by the collector, not by lifecycle. That leg stays open. Offline-proven: stt-faults.test.ts (storm→one summary per kind, counts, backend detail preserved, terminal-only, healthy meeting unchanged, a throwing reporter never perturbs the exit) + meeting-api test_lifecycle_diagnostics (persisted to data + webhook, negative control). Bot suite clean, gates green; meeting-api 770 pass (the 31 calendar-sync failures are pre-existing on main, 768 pass there).
…of publishing a stale draft flushSpeaker(force) — the bot leaving, a meeting ending, a channel rotating — emitted buffer.lastTranscript and reset, so audio newer than that draft was discarded. The emitted segment still carried the buffer's FULL extent, so a turn whose draft covered only its opening seconds went out as a full-length segment with most of its words missing and nothing downstream saw a gap. The inFlight->pendingFinal guard that exists for exactly this case sat below the draft branch and was unreachable whenever a draft existed, which is the common case. Owed audio is now submitted (or deferred to the in-flight response) before the draft is considered; the draft stays the fallback for an empty or hallucinated final response, so a turn is still never lost outright. Also adds the transcript QUALITY oracle this was found with: speech_fixture.py builds a captured-signal.v1 session from real speech with KNOWN text, quality.test.ts replays it through the real lane against real STT and scores recall/precision/fragmentation/attribution against that truth. Every existing replay harness used a mock transcribe returning canned strings, so none could see a transcript that was structurally perfect and substantively wrong. Controlled measurement (q2 fixture, local whisper, STT pinned +3000ms, zero dead air, n=4 each): baseline recall 0.777 0.777 0.489 0.628 | lastTurnRecall 0.976 0.976 0.146 0.146 fixed recall 0.989 x4 | lastTurnRecall 0.976 x4
…is stored once The store upserts on (meeting_id, segment_id) — a segment_id IS the identity of a line of transcript. The mixed lane published its forming tail under turn:N:p<i> (indexed off the UNCONFIRMED slice, so it renumbered as the turn advanced) and the confirmation under turn:N:<seq>. Two identities for the same words: the draft row was never replaced, so every confirmed sentence left its draft behind as a permanent second row. Observed in a real Jitsi meeting — 6 stored rows for 3 spoken sentences: turn:54:p0 | происходящего. turn:54:0 | происходящего. turn:54:p1 | И, мало того, ... turn:54:1 | И, мало того, ... The forming tail now carries the ids it will confirm under (turn.seq is already past this pass's confirmations), so draft -> confirm is an upsert on one row. This is what the gmeet lane already does; the mixed lane is now the same. Nothing consumed the 'p' prefix. Red test models the CONSUMER, not the wire: every published segment folded into a map keyed by segment_id, last-write-wins — exactly upsert_segments — then asserts no sentence is stored twice. Pre-fix that store held 7 rows for 4 sentences; post-fix 4.
…turned on The recorder shipped in 0.12.15 and the bot honours invocation.v1 captureSignalEnabled, but NOTHING set it: bot_spawn did not know the option existed and no deploy surface exposed it. The feature was unreachable in every real deployment. The live runs that 'proved' it worked had the env var set by hand on containers launched OUTSIDE the control plane — which is precisely what hid the gap. Adds a deployment-scoped CAPTURE_SIGNAL_ENABLED (compose + helm, config.v1 declared) that flows knob -> invocation -> bot, and a test that pins the reachable path rather than the unit in isolation. OFF by default on purpose: it persists RAW meeting audio, a stronger retention claim than a transcript, so turning it on is an explicit operator decision. Caught while committing: the helm template read .Values.services.captureSignalEnabled while values.yaml declared it under meetingApi — there is no 'services' section, so the knob would have rendered 'false' no matter what an operator set. The same dead-config bug the change exists to fix, one level up. gate:config-contract checks that a declared key appears on a deploy surface, not that the .Values path it references resolves, so it stayed green. Verified by rendering: --set meetingApi.captureSignalEnabled=true now yields "true", default yields "false".
It fed the lane minAudioDuration 0.15 / submitInterval 0.1 — 13x and 20x faster than the production defaults (2s/2s) — inherited from the batch replay test. Every latency number it produced therefore described a configuration no deployment runs. It now uses the production defaults, overridable via BOT_SPEAKER_* to compare tunings deliberately. The budget also moved from total time-to-text to PIPELINE OVERHEAD (total minus submissions x STT). Total is dominated by the STT gateway, which this repo does not control: measured against production 2026-07-19, a 4s chunk costs p50 ~2.1s server time with only ~370ms of that network. An earlier version 'passed' a 1500ms total budget purely because the mock STT was 250ms — a green that proved nothing about the product. On the corrected config the harness is RED and should be: overhead p95 3954ms against a 900ms budget, projecting ~8.15s time-to-text at production STT. That matches the user-reported 'time to first transcript is very high' and is the baseline for the gmeet cadence work. The harness is opt-in (npm run replay:paced) and gates nothing — gate:replay runs replay/replay-mixed.
…sion Replays a recorded captured-signal.v1 session — its audio, its speaker hints, and production's OWN segmenter cuts — through the REAL mixed lane and scores what survives: coverage, word retention (words STT returned vs words published), segment size distribution, holes, duplicates. MIN_TURN_MS sweeps boundary coalescing so the over-segmentation axis can be measured rather than argued about. First reading, on a live jitsi session sharing a YouTube tab (continuous speech): 183 recorded cuts in 190s, p50 inter-cut gap 0.41s, 142/182 under 1s, every confidence 0.35-0.47 STT windows p50 0.77s, 34/54 under a second -> Whisper answers 'Thank you.' to a 0.38s sliver word retention 1.000 — the pipeline publishes every word STT hands it That retention number retires a hypothesis I had argued from code reading: that mixed LocalAgreement-3 was dropping text because turns close before they can confirm. It is not. The loss is upstream — continuous speech is diced into sub-second windows, so what STT returns is filler rather than transcription. MOCK_STT=1 is deliberate here and scoped: the question is structural (words in vs words out), so a stand-in emitting a known word count per second measures retention exactly, deterministically, and without a 10-minute CPU-whisper round. It says nothing about ASR quality and must not be used to claim any.
…ic set, two loops, stage-attributed fixtures FRAMEWORK.md is the governing doc: six stages (capture -> segmentation -> STT -> assembly -> publish -> store), an invariant metric set for every lane, the external/internal loop contract (no fix without a fixture from a real session; no ship without human re-witness), the ground-truth ladder, and the anti-patterns of the calibration day encoded as rules. Instruments landed with it: - eval/src/tape-to-signal.mjs — desktop raw tape -> captured-signal.v1. THE bridge: a human-witnessed session becomes an offline, scoreable fixture instead of something only replayable into a running desktop. Prints capture duty cycle on conversion, so a fixture self-reports whether its audio was even delivered before anyone blames a pipeline. - eval/src/single_pass_truth.py — the SAME audio the pipeline received, sent to the SAME STT in one pass, as the reference. Splits every miss into model ceiling (absent from the reference too — not ours) vs streaming loss (present there, missing live — ours, quantified). - desktop: VEXA_STT_TAP=1 logs each submission window + its answer (a transcript hole is ambiguous without it), and the gmeet lane now runs the SHIPPED cadence via BOT_SPEAKER_* (production defaults 2/2/2) instead of a hardcoded desktop-only 1.5/3 — the hot loop must measure the configuration that ships. - extension README: corrected 'Google-Meet-only by design' — src/meeting.ts detects and captures gmeet/youtube/zoom/teams; a YouTube tab is the cheapest repeatable mixed-lane source there is. Calibration baselines (2026-07-20): jitsi bot capture duty cycle 65.0% (capture defect; segmenter innocent — inter-cut p50 0.41s tracks the delivery-gap p50 0.37s), youtube-direct control 97% duty / 0.671 coverage / 0% filler; hallucination is monotonic in window size (31% filler <1s, 0% >=2s), which names #613's un-root-caused sibling half.
… and ten audited gaps Attribution was the thinnest part of v1 and is now written down properly: ClusterNameBinder is THE binding logic and the four platform speaker adapters are thin sources feeding it (hintKindForPlatform: teams -> dom-outline, else dom-active), each adapter shared by BOTH the bot and the extension. So validating the binder on ONE mixed platform validates it for jitsi, teams and zoom; only hint quality stays platform-specific. Adds attribution's own stage chain, the four signals that need no ground truth (provisional rate from transcript.v1 , hint matched/missed from onHintOutcome, rename churn, speaker cardinality), and the two truth-bearing oracles — including self-identifying speech, which puts the speaker label INSIDE the content so the single-pass reference can score attribution without any labelling. Also converts claims into a coverage matrix whose blanks are honest (zoom/teams entirely unmeasured; gmeet content measured only on a TTS fixture), and records ten gaps this framework does NOT yet close. The sharpest is G1: single_pass_truth.py is speaker-BLIND, so a transcript with perfect words and scrambled speakers scores 1.000/1.000 — mock-blindness repeating on a different axis. G2 admits the attribution metric row was aspirational. Others: an uncalibrated reference (G3), no fixture corpus at all (G4 — today's fixtures live in a temp dir), nothing in gates (G5), no store-fidelity metric where the false '64.9% dropped' came from (G6), latency unmeasured live (G7), English-only while the originating complaint was Russian (G8), no overlap metric (G9), no single session bundle (G10). Also fixes the tape converter to discriminate recording chunks by their REC1 magic — a recording chunk decoded as an audio frame yields plausible garbage (magic reads as speakerIndex, media bytes as denormal timestamps), which silently corrupted the first conversion's duty cycle; and clips the live transcript to the recorded session's own window before scoring, which moved measured precision from 0.841 to 0.941.
… G11, not a pass The coverage matrix let gmeet's content cell read as if a TTS fixture covered it. It does not: the mixed lane's recall .905 was measured against a single-pass reference on a real session, while gmeet has only ever been scored on synthetic known-text audio. The lanes do not transfer — LA-2 vs LA-3 confirm economies, per-participant <audio> capture vs the bot's webrtc-hook chain, and gmeet-only loss surfaces (glow gating #616, the silence gate with no env knob). Also marks gmeet's capture cell as a path independent of #850's, so the bot mixed-capture fix is not mistaken for covering it, and records that the 0.12.15 attribution behaviour the maintainer attests to has never been written down as a number a regression could fail against. #851 scope broadened to match: loss + latency + a no-regression sweep, with the #613 pause repro walked on gmeet so post-pause loss is either fixed or attributed to #616 with evidence.
…t regression test (#848) FRAMEWORK.md claimed "every witnessed session joins the regression corpus" and there was no corpus: no location, no naming, no index, no per-fixture expected values. A session file alone cannot be a regression test — replaying audio proves nothing unless the numbers it USED to produce are stored beside it. The day's two calibration sessions were sitting in /tmp. An entry is $VEXA_CORPUS/<platform>/<slug>/: the session, a baseline.json holding every metric at promotion time, a manifest.json pinning provenance (source file, window, commit, STT service and model, sha256), and optionally the single-pass reference and the live transcript it was scored against. CORPUS.md is the tracked index, so a fresh checkout finds the corpus even though the audio is deliberately not in the repo. promote-fixture.mjs tape or captured-signal session -> entry, baselines computed score-fixture.mjs re-measure every entry, diff against its baseline, exit 1 on drift signal-metrics.mjs the truth-free delivery/shape metrics, one definition for all callers Three metric blocks, and the difference between them is the point. Delivery/shape and content recompute from stored bytes, so drift there means a scorer or the fixture changed. The lane block re-runs the REAL mixed pipeline over the fixture with a mock STT — everything held fixed except our code, so movement is the code moving. tape-to-signal grew --head-sec because a tape keeps growing after the session someone measured ended; the youtube entry re-derives byte-identically from a tape twice its length. The lane harness now models the CONSUMER: publish AND publishPending upserted by segment_id, last write wins, exactly upsert_segments. The first version of that model dropped the `pending` argument production actually ships and so could not see a draft-identity defect at all — the instrument was blind to the defect it was built to catch. Fixed, and reverting 4c030cd now takes youtube/2026-07-20-continuous-speech from 11 store rows / 0 duplicates to 22 rows / 11 duplicates. Fixtures with no recorded cuts (any desktop tape) run the real PyannoteSegmenter rather than scoring one 20-minute turn and calling it the lane. single_pass_truth.py reads gzipped sessions, emits its score as JSON for a baseline, no longer demands STT credentials to re-score against a reference it already has, and accepts a service URL with or without the route on it. Recorded, both entries at 5b13eff: jitsi/2026-07-20-capture-gaps duty 0.650 · recall 0.858 · precision 0.723 youtube/2026-07-20-continuous-speech duty 0.949 · recall 0.905 · precision 0.941 Same source material, same lane, same STT — the gap is the bot's capture chain and nothing else. Closes framework gap G4; G6 partly.
The corpus entry jitsi/2026-07-20-capture-gaps delivers 136.2s of audio over 209.5s of wall clock —
a 65.0% duty cycle against 94.9% for the extension on the same source material. Two mechanisms
produce that identically and have unrelated fixes:
* the peak-amplitude silence gate (mixed-audio.ts) refuses a whole 256ms buffer below 0.005
* createScriptProcessor runs its callback on the MAIN THREAD and drops buffers when the page is
busy — documented in this repo by the module written to replace it (gmeet-capture/pcm-capture.ts),
and the bot's renderer is the busiest we run: four AudioContexts, a software-rendered meeting
client, and a concurrent Opus encode
Nothing distinguished them. The delivered frames' peak amplitudes are truncated exactly at 0.00503
with none below, which proves the gate is live — but a buffer the processor never hands over is
never peak-tested either, so that evidence is equally consistent with both. The gmeet lane logs
seen/emitted per stream; this lane logged nothing per frame.
stats() measures both against the AudioContext's own clock, which is the only local witness to how
much audio existed: processorDeficitSec = rendered - delivered, gatedSec = what the gate refused.
Counted in samples rather than buffers, so the figures stay true whatever buffer size the engine
picks. The bot now passes a log so a live run reports the split; the extension already had a quiet
default and keeps it.
Tested against the existing browser shim with a test-driven context clock — advancing it
independently of the buffer callback is exactly how a starved processor is modelled, and the test
asserts the gate is NOT blamed for a processor deficit and vice versa.
Diagnosis and its correction are on #850. No fix yet: which mechanism dominates is now a
measurement, and the next live run makes it.
…as holes (#850) cut() concatenates only the frames that exist, so a span containing a hole yields audio SHORTER than the span it claims. Whisper describes that compressed audio in its own timebase, and the mapping read it as `spanStart + ws.start*1000` — treating compressed time as wall time. Every word after a hole was stamped early by the accumulated hole, compounding across the turn. The cut's own comment asserted frames are contiguous; on this lane they are not. Holes are not exotic and not someone else's bug: capture always has some (a dropped buffer, a gated silence, a renegotiation). On jitsi/2026-07-20-capture-gaps, 108 of 183 recorded segmenter cuts land at a wall time where no audio was ever delivered. Downstream, turns open and close against the wrong instants and the hint binder matches names against a clock that has drifted — so this decays attribution, not only timing. cut() now returns its span layout and wallTimeAt() maps compressed seconds back through the holes. The audio stays compressed: zero-filling would restore clock equality at the price of inviting the model to hallucinate over manufactured silence, and would cost STT time for nothing. Both edges resolve a boundary correctly — a start opens the next run, an end closes the previous one — so a sentence is neither stretched across the hole after it nor pulled back into it. Also: a span may no longer reach behind the ring. cut() can only return what it holds, so a span older than RING_MS returned RECENT audio under an OLD start, placing an entire turn at the wrong instant. Clamping spanStart to the oldest surviving frame keeps the claim and the samples the same age. Red test (model-free): speech, a 3s hole no frame ever covered, speech again. Pre-fix the post-hole sentence published at 2000ms — inside the hole, 3s early. Post-fix [0-2000] and [5000-7000], both where the audio actually was. Two instrument corrections this surfaced, both of which invalidated numbers I had already reported: * the replay harness fed frames in a tight synchronous loop, so the WHOLE session was fed before the lane's async pump ran once and audio was evicted from the 120s ring before anything read it. The run scored the leftovers. It now yields between frames: submissions go from p50 0.25s (an artifact) to p50 2.56s (production-like), 185 STT calls to ~435, 11 publishes to ~279. * `retention` under MOCK_STT is not a loss metric. The mock invents fresh tokens per call, so every LocalAgreement resubmission inflates the denominator with words that were never new. Published words are stable run to run (2746-2750) while sttWords is not (3935-4138). It is now recorded only against real STT. Lane baselines re-recorded with tolerances measured from five consecutive runs rather than assumed exact: structure (coverage 0.905-0.906, holes 6/6/6/6/6, words 2746-2750) is stable; call-count figures are not. What the corpus exists to catch moves far further — reverting 4c030cd still takes storeDupes 0 → 11.
…850) Two numbers reported as measurements on 2026-07-20 were artifacts, and neither was visible in a single run: a replay that fed a synchronous ring faster than the async pump could read it (scoring the leftovers, with a submission p50 of 0.25s against production's ~2s), and a retention ratio whose denominator the mock STT controlled (every LocalAgreement resubmission counted as words lost). Both are now rules in FRAMEWORK.md rather than history: repeat a run before baselining it; a ratio whose denominator the instrument controls is not a measurement; a replay must not outrun the code it replays, and a cadence that does not resemble production's indicts the instrument first. CORPUS.md carries the re-recorded lane baselines, the five-run spread the tolerances come from, and the caveat that the single-pass comparison is built from DELIVERED audio and is therefore blind to capture loss by construction.
) The framework's attribution row was aspirational (gap G2): every signal was already emitted and none was read. `onHintOutcome` is the clearest case — #797 reports hints "detected, silently discarded", which is precisely this counter going unwatched since it was added. Four signals, none of which needs a labelled meeting: provisionalRate share of stored WORDS still under a seg_N cluster id — a turn no hint ever claimed hintMissRate the binder's own per-hint verdict, finally counted renames/churn turns whose name settled somewhere other than where it started (A→B→A is defective whichever name is right, so distinctness is the measure) cardinality published speakers vs distinct hinted names Scored against the store, not the wire: `rename` repaints rows in place, so the attribution a reader ends up with is the LAST name each row was given. First measurement, jitsi/2026-07-20-capture-gaps: 16.7% of words provisional, and 79 of 86 hints missed — a 91.9% miss rate. That second number is mostly NOT the binder. Of 83 in-window hints, 55 (66%) point at a wall instant where capture never delivered a frame, so no turn could exist to match them; 3 more fall outside the audio window entirely. Capture loss is upstream of attribution loss, which reorders #847's chain: #850 is not parallel to the attribution work, it gates it. What remains after subtracting the holes is 21 hints that pointed at delivered audio and still missed — a real residue, and the honest size of the binder-side question. A fixture with no hint stream (a shared tab) reports attribution as n/a rather than 100% provisional: nobody to name is not the same as failing to name. Tolerances follow the lane block's measured jitter. What these exist to catch — a namer that stops naming, a hint stream that goes dark — moves in tenths.
Coverage matrix carries the first attribution numbers (jitsi: 16.7% provisional, 91.9% hint miss) and G2 closes for the mixed lane. What stays open is named rather than implied: the gmeet lane binds at capture from the glow and needs its own reading, and these four signals score WHETHER a name was assigned, never whether it was right — that needs the self-ID oracle.
… suspect (#850) Two mechanisms can omit a capture frame and they leave identical evidence in a recording, because a buffer the ScriptProcessor never delivered was never peak-tested either. The code map ranked the main-thread ScriptProcessor first — this repo documents that node as dropping and duplicating buffers under load, in the very module written to replace it, and the bot's mixed lane is the last path still on it inside the busiest renderer we run. Measurement says no. Driving the real capture module in real chromium, 15s per arm: continuous tone, idle page duty 99.0% gated 0.0s processor deficit 0.1s speech-like + digital silence, idle page duty 99.3% gated 2.6s processor deficit 0.1s continuous tone, main thread BUSY duty 99.1% gated 0.0s processor deficit 0.1s speech-like + silence, main thread BUSY duty 100.7% gated 2.8s processor deficit 0.0s The BUSY arms block the main thread for 180ms out of every 220ms — heavier than a meeting client imposes — and the processor still delivered every buffer (58 seen, 58 emitted, 15.0s rendered). A blocked main thread queues those callbacks; it does not discard them. The gate, meanwhile, removes precisely the digital silence and nothing else. So the ranked suspect is refuted and the silence gate stands alone as the mechanism, which is the distinction that decides the fix: a gate is a policy to change, a starving processor would have been a rewrite onto the AudioWorklet. What this does NOT establish: that the bot's renderer behaves like the bench's. A synthetic block is not four AudioContexts, a software-rendered meeting client and a live Opus encode. The counter now ships in the bot, so any live run reports its own split and can contradict this.
…ency tail offline (#850) A live desktop session against the hosted STT (transcription.vexa.ai, whisper-1) showed round trips of p50 1419ms and max 9045ms — but the service is not the interesting half. 1217 submissions, grouped into runs where a turn re-sends a longer span each pass: 352 growth runs · p50 3 passes / 5.3s · max 11 passes / 35.0s 5100 audio-seconds SENT for 2266 seconds of distinct span = 2.3x re-transcription Three passes is LocalAgreement-3 working as designed; the span only advances when something confirms, so the unconfirmed window is re-transcribed every pass. The tail is the problem: an 11-pass turn reaches a 35s window, and a 35s window costs seconds of STT before a single word can appear. resendRatio and maxSubmitSec now ride in the lane block. The youtube corpus entry reproduces the tail with no meeting: maxSubmitSec 33.1s against the live session's 35.0s, resend 1.5x. So the latency defect is now a fixture, not an anecdote — which is what makes the fix testable before it is witnessed. No fix here. Bounding the window is the obvious lever and it is not free: a cap that is not paired with guaranteed forward progress stalls a turn that will not confirm, and forcing a confirm trades latency for the accuracy LocalAgreement exists to protect. That tradeoff gets swept on the corpus rather than guessed.
youtube/2026-07-20-witnessed-panel — the maintainer watched this session live in the extension panel and judged time-to-text ~6s and quality good. It is the only entry carrying a human verdict, so it is the point the instrument numbers are calibrated against rather than argued from. It also anchors the far end of the delivery curve: capture duty cycle 100.0%, zero gaps over 100ms in 557 seconds, on the same capture code that delivers 65.0% through the jitsi bot. Whatever removes 35% there leaves this source untouched — which is the argument for changing the silence gate rather than the capture chain. Content is not scorable for this entry and the manifest says so: the desktop store had restarted, so no live transcript survived to compare against a reference. Recorded as a gap in the entry rather than quietly omitted.
…d no labelling (#849, #852) The four truth-free signals say whether a name was assigned. They cannot say whether it was correct, and that gap was being carried as "zoom will start it" — a plan that needs a meeting, real people and a human to judge. It does not. speech_fixture.py grows a `--lane mixed` mode: the same known-text TTS clips, but mixed to ONE stream with the speaker withheld from every frame (the gmeet lane names frames because capture genuinely knows; every other platform does not), and the names delivered out-of-band as a hint stream. Because the fixture knows who spoke when, the published transcript can be scored for attribution ACCURACY — the truth-bearing oracle FRAMEWORK.md listed and nobody had built. The hint dials are the point: --hint-lag-ms, --hint-drop, --hint-wrong reproduce the three ways a DOM name source fails, seeded so a fixture is reproducible. 3 speakers, 9 turns, 355 known words: hint stream accuracy given a good hint provisional wrong rows clean (250ms) 91.3% 91.3% 1.2% 1 lag 750ms 82.6% 82.6% 1.2% 3 30% dropped 65.2% 93.8% 22.2% 3 34% wrong 47.8% 84.6% 0.0% 12 Two things fall out. The binder is largely faithful — "given a good hint" holds 82-94%, so its errors are dominated by the quality of what the DOM tells it, not by the binding. And the last row is the one that matters: half the transcript under the wrong speaker while EVERY truth-free signal reads healthy — 0.0% provisional, 3 published against 3 hinted, nothing unnamed. Recorded in FRAMEWORK.md as G1a, demonstrated rather than asserted. Accuracy carries the same run-to-run jitter as the rest of the lane block (the clean arm scored 87.0% and 91.3% on two runs), so it is a ±4pp instrument, not a ±0.1pp one. Enough to separate 91% from 48%; not enough to chase a point.
…d on merit (#850, #854) Reverting the segmenter-timeline change exposed this: the corpus could tell me a diff moved coverage 0.905 -> 0.835 and slivers 78 -> 56, and could not tell me whether that was a regression or a correction. A corpus entry's stored transcript came from the ORIGINAL live session, so it scores that session and nothing else. Every content question about a diff was unanswerable, and the honest response was to revert a change I still believe is right. TRANSCRIPT_OUT writes what the CHANGED code produces from the same audio, in the shape single_pass_truth.py --realtime already reads. score_replay.py scores it against either a corpus entry's single-pass reference, or — on a synthetic fixture — the clip text itself, which is absolute truth rather than another model's opinion. Proven end to end on the 3-speaker mixed fixture with real STT (local faster-whisper-small): KNOWN TEXT: 355 words · REPLAY: 351 words recall 0.907 (322/355 kept, in order) · precision 0.917 (29 words never said) And with real STT rather than the mock, attribution accuracy on that fixture reads 96.4% against 87-91% under MOCK_STT — the mock's uniform windows cut turns differently, which is one more reason its numbers are structural only. The metric set is now complete on a fixture that needs no meeting and no human: delivery, content against absolute truth, attribution accuracy AND the four truth-free signals, integrity, shape, and re-transcription cost.
, #851) I claimed gmeet-lane attribution was unscored. It was not — quality.test.ts has always scored "segments under the right speaker"; nobody had run it on a multi-speaker fixture and written the number down. Corrected by measuring rather than by asserting again. Same 3 speakers, same known text, same local STT: gmeet (per-channel, named at capture) attribution 1.000 (14/14) · recall .873 · precision .963 mixed (one stream + DOM hints, clean) attribution 0.964 · recall .907 · precision .917 mixed (hints wrong 34% of the time) attribution 0.478 This is the layer model's central claim, finally priced: naming at capture is free and cannot be got wrong, while the mixed lane's attribution is only ever as good as the hint stream feeding it. It also says where per-platform effort belongs — not in the binder, which holds 82-94% given a good hint, but in whether each platform's DOM produces one. Coverage matrix updated; the gmeet row's content and attribution columns are no longer blank.
…ign actually costs (#847, #854) Every content number in the corpus rested on an untested assumption: that a single-pass reference is closer to truth than the live transcript. If it were not, recall against it would measure agreement between two flawed transcripts and mean nothing. G3 named that and nobody had settled it. A synthetic fixture settles it, because the spoken text is known absolutely rather than inferred: SINGLE PASS vs known text : recall 0.977 precision 0.943 <- the model's ceiling on this audio OUR REPLAY vs known text : recall 0.907 precision 0.917 <- model + streaming together streaming's OWN loss: 7.0 points of recall, 2.6 points of precision So the reference is substantially better than the pipeline, recall against it is loss rather than agreement, and the corpus's content axis means what it claims. G3 closes; what remains of it is narrower and now stated as such — 60s chunking with 3s overlap can still duplicate or drop at the seams, and this bounds the total error without isolating them. It also answers "how far from perfect, and how much of that is ours": of the 9.3 points between the pipeline and the known text, 7.0 belong to the streaming design and 2.3 to the model.
…al meeting meeting-speaker.mjs joins a Jitsi room in headless chromium with a WAV as its microphone (--use-file-for-fake-audio-capture), so a real WebRTC source with known ground truth exists without a person, a TTS service, or a host. Two of them plus a real bot is the shape that reads the capture delivery split from a production renderer, which is the one measurement the headless bench cannot make: the bench proved the ScriptProcessor survives 82% main-thread blocking, but a bot's renderer is four AudioContexts, a software-rendered meeting client and a live Opus encode. Adapted from the lab script; playwright is imported by path because eval is deliberately not a workspace member. First run did not reach the measurement, and the reason is worth recording: the speakers join with `#config.prejoinPageEnabled=false` and land in the conference, while the bot joins plain https://meet.jit.si/<room> and sat in "waiting for admission" for 234s. meet.jit.si gates a joiner arriving into a room that is already populated. The bot must therefore join FIRST and let the speakers arrive around it — an ordering constraint on autonomous multi-party runs, not a bot defect.
….si nobody can meeting-speaker.mjs grows ADMIT=1: poll for a lobby knock and let it in, so an autonomous multi-party run does not need a person to click Admit for every bot. It cannot work on meet.jit.si, and the evidence is worth keeping. Spawning the bot into an EMPTY room still logs "[Jitsi] Bot is in the lobby — waiting for a moderator to admit", so the earlier guess that this was a join-ordering problem was wrong. Joining a speaker to admit it fails the same way, and the speaker's own page says why: "Asking to join meeting… The conference has not yet started because no moderators have yet arrived. If you'd like to become a moderator please log-in." meet.jit.si requires an AUTHENTICATED moderator to start a conference, so every unauthenticated participant queues — the would-be doorkeeper included. No amount of ordering or polling escapes it. The dial stays because it is correct against a Jitsi we control, which is the actual fix: a self-hosted host has no lobby, and the compose stack already parses declared jitsi hosts. Until then, a live multi-party jitsi run needs one authenticated human to open the door — the one step in this harness that is genuinely not automatable on a public deployment.
…s discarding time (#850) Measured on a real bot renderer, in a live meeting, with the delivery counter: seen=500 emitted=168 · delivered 128.0s of 128.0s rendered · processor deficit 0.0s · gated 85.0s Every buffer the graph rendered reached the callback — in the renderer that was the entire basis for suspecting otherwise (live Jitsi client, software-rendered, four AudioContexts, concurrent Opus encode). So the gate was the whole capture deficit, and the ScriptProcessor hypothesis is dead in production as well as on the bench. Dropping a frame does not drop audio, it drops TIME. The segmenter splices speech onto speech across the hole and reads it as a speaker change; turn boundaries land where nothing was delivered (108 of 183 recorded cuts on jitsi/2026-07-20-capture-gaps); the hint binder matches names against a clock that has drifted (55 of 86 hints point into holes). What the gate saved is already saved twice downstream — the pipeline refuses to submit a span below DROP_RMS, and the post-Whisper acoustic gates drop what comes back. A non-positive threshold now disables gating outright. `maxVal > 0` would still have refused an all-zero buffer, and an all-zero buffer is exactly what a codec's silence suppression emits — the one case a disabled gate most needs to pass. Pinned by a test. The extension's two MIC-path gates are deliberately untouched: the evidence here is from the meeting path, and changing the mic on the strength of it would be the speculative fix this issue has spent its whole life avoiding. NOT YET VALIDATED, and it cannot be validated offline: every corpus fixture was recorded AFTER the gate, so its holes are baked into the bytes and no replay can show them filled. This needs one live capture on a platform we ship.
…uman (#850) New corpus entry jitsi/2026-07-20-gate-off: the same bot capture chain as 2026-07-20-capture-gaps with the silence gate off, two synthetic speakers on an open Jitsi host. Live counters held seen=500 emitted=500 · processor deficit 0.0s · gated 0.0s for the whole run. duty cycle 0.650 -> 0.999 gaps >100ms 199 / 115.1s -> 9 / 1.9s inter-cut p50 0.41s -> 1.47s cuts under 1s 142 -> 10 provisional words 16.7% -> 0.0% Both knock-ons predicted from the hole analysis are visible. The segmenter stops cutting on holes — 8.7x fewer cuts — which was the mechanism starving LocalAgreement and feeding Whisper the sub-second windows it hallucinates over. And attribution improves with the binder untouched, because the 55 of 86 hints that pointed at instants capture never delivered now have turns to bind to. Read the duty cycle, not the shape deltas: the red arm is continuous YouTube audio and this one is sparse TTS clips, so the arms are not matched. What holds either way is that a CONTINUOUS source scoring 65.0% was the defect. Autonomy: jitsi.dorf-post.de logs "Bot immediately admitted (no lobby)" where meet.jit.si demands an authenticated moderator, so this whole run needed nobody. Unmeasured and worth knowing before production: frames now flow during silence over a hop that ships each one as a 4096-element JSON array.
… the incoming one (#852) Found in the first real-platform fixture. zoom/2026-07-20-live-zoom, a live 5-person call: 142 hints · 13 speaker switches · ZERO isEnd events 2 speakers published against 5 hinted Zoom's adapter reports who is lit NOW (`name | null`), unlike Teams and Jitsi whose adapters emit speaking start/end events themselves — so only this lane had to derive the end edge, and it derived it wrong: if (name) hint(name, t, false); else if (lastActive) hint(lastActive, t, true); // only when NOBODY is lit A real meeting hands over directly, A -> B, never passing through "nobody lit". So A's window never closed, and the binder names a turn by max-overlap against hint windows — an open window keeps winning against every speaker who follows it. Three of five participants were attributed to someone else, while every truth-free signal except cardinality read clean (0.0% provisional, nothing unnamed): the exact blindness recorded as G1a. The transition rule is now a pure function (makeActiveSpeakerBridge) so it is provable without a page, and pinned by a test: a handover closes one window before opening the next, a heartbeat re-assert of the same name does not close it, and the close lands before the open so two windows never overlap. Not yet validated live — it needs another Zoom call to confirm 5 hinted speakers become 5 published ones. The mechanism and the fix are both offline-provable; the outcome is not.
…aker (#849, #852) The signal compared distinct hint NAMES against distinct published speakers. A hint stream is a poll, so anyone briefly lit counts as a full participant — and on zoom/2026-07-20-live-zoom that produced "3 of 5 speakers never reach the transcript", which I reported as a defect and then had to withdraw: Joyful Jen 226.0s lit Anthony 46.0s Rick 6.0s Antee 4.0s Tony 2.0s Twelve seconds between the bottom three, of a 269s call where two people did 98% of the talking. A two-speaker transcript is the correct outcome. Consecutive polls on one name now merge into a lit window, and only names lit at least 8s count toward the expected cardinality; the rest are reported separately rather than silently. The same entry now reads "2 published vs 2 substantively hinted (+3 lit under 8s, not counted)". This is the third over-claim of the day and the only one whose cause was the instrument rather than the pairing of artifacts. The pattern behind all three: a red number was treated as a defect before asking whether the EXPECTED value was right. The two guards now in the tree — a transcript must match its session's meeting id, and cardinality must weight by lit time — both attack that.
…rned into a URL (#501) Production, 2026-07-20: 8 of 34 visible bot pods in Error, 6 of them "failed to join", spread across the day. Every one navigated to https://teams.microsoft.com/l/meetup-join/389141384648269 — a bare numeric id — and ended up on login.microsoftonline.com/common/oauth2/v2.0/authorize, with the Teams app reporting `[_getUserIdentifier] user missing`. A joinable Teams deep link is …/l/meetup-join/19:meeting_<b64>@thread.v2/0?context={Tid,Oid} — the thread id AND a context naming tenant and organizer. The repo's own parser says so (collector/meeting_link.py:71). A numeric meeting id carries neither, so the constructed URL is syntactically fine and semantically meaningless, and Teams redirects to authenticate. Everything after that is noise the logs make look like a selector problem: each pre-join step reports its control "not found" because it is probing a login form, and admission polls 31s and finds no admitted/rejected/lobby indicator because there is no meeting page. I first mis-attributed these failures to the gmeet locale issue (#856) on the strength of that final line alone. The rule was already written, one comment above the template that broke it: # NO jitsi template: a jitsi room name is scoped to a DEPLOYMENT … constructing a URL from the # bare id would silently join the public room of that name — the wrong meeting It applies harder to Teams: a jitsi room name at least resolves to SOME room, while a Teams id without its thread resolves to nothing. construct_meeting_url now returns None for a teams id that is not a thread id, which routes into the typed 422 that already existed — and the message names the id as the problem rather than claiming the platform is unsupported, so the caller is not sent to fix the wrong thing. Cost this removes: a full container lifecycle, 31 seconds of polling, and a silent failure the caller could not distinguish from a meeting with no speech. Suite green in the project's own environment: `uv run pytest` gives 807 passed, 5 skipped, 0 failed. (An earlier run of mine reported 31 failures in test_calendar_sync.py; that was system `python3 -m pytest` against mismatched deps, not a real result — the gate runs uv, and is green because the tests actually pass.)
…g quiet (#852) Live bot in a real Zoom meeting with two known-text speakers, 2026-07-20. Content transcribed correctly; attribution 100% provisional — seg_0, seg_4, seg_7 — and the bot's own counters read hint-counters bridge-crossed=0 pipeline-received=0 binder-matched=0 binder-missed=0 Zero hints ever left the page. Probing app.zoom.us/wc/<id>/join the same way the bot joins, ~25s after entering the meeting: .speaker-active-container__video-frame 0 .speaker-bar-container__video-frame--active 0 .video-avatar__avatar-footer 0 Every selector the Zoom name path depends on matches nothing; the only class anywhere near the video area is audio-voip-active-icon. The DOM this adapter was written against is gone. What made it expensive is not the rot but the SILENCE. The watcher logs only on transitions, so one that never sees a speaker emits nothing: no error, no warning, a clean bot log, a complete transcript, and all-zero counters that are indistinguishable from a meeting where nobody spoke. The extension path on the same platform produces 142 hints over 5 real names, so the module works — it just cannot say when it has stopped working. It now reports having seen no speaker at all, and names which cause the DOM supports: containers present but unnamed ("the room may be silent") versus containers absent ("NONE of the containers exist … selectors are stale, attribution WILL be empty"), listing the selectors that missed so the fix does not need a source dive. Tested both ways against a shimmed DOM. NOT fixed here: the selectors themselves. That needs a survey of the current client to find where the active speaker is now expressed, and audio-voip-active-icon is the lead. This change is what makes the next redesign visible in the first minute rather than after someone reads a bad transcript, and the teams and jitsi watchers share the shape.
…854) I read a replay's coverage of 0.378 on zoom/2026-07-20-live-zoom as a product defect — 145.8s of a 269.1s session apparently never reaching STT, with capture having delivered all of it at duty 1.001. Checking the LIVE session the fixture came from, before writing it up: LIVE 68 segments · union coverage 267.7s over a 331.2s span = 0.808 REPLAY coverage 0.378 (101.5s of 268.8s) More than 2x apart, so the replay figure is the harness. There is no 54% loss. quality-mixed.test.ts states this in its own header — "replay is unpaced … the submitted-window sizes and every latency derived from them are the harness's, not production's" — and coverage sits downstream of submission cadence, so it inherits the distortion. I wrote that warning and read past it. CORPUS.md now says so above the entries and marks the figure at each one; score-fixture prints it with a `~`. The number stays in the lane block because it is a perfectly good regression detector against the same fixture's own baseline — it just cannot be read as "the lane covers X% of a meeting", which is exactly how I read it. The live loss figure unaffected by any of this: recall 0.905 / precision 0.941 on the youtube control, measured against a single-pass reference rather than derived from a replay.
…already does (#852) A live bot in a real Zoom meeting produced a correct transcript attributed entirely to seg_N, with hint-counters bridge-crossed=0 — no hint ever left the page. The maintainer's observation is what made this findable: the extension and the bot use the SAME module (clients/extension/src/inpage.ts imports createZoomSpeakers from @vexa/zoom-capture), so the adapter cannot be the difference. Looking at how each side installs it, the bot has an internal asymmetry: browser-utils.global.js context.addInitScript → re-injected into EVERY document installRemoteAudioHook context.addInitScript → re-injected into EVERY document the speaker watchers one-shot page.evaluate → setInterval dies with ITS document So audio and names have different lifetimes. Zoom's web client navigates around join (the URL picks up _x_zm_rtaid= after the Join click, observed directly while driving speaking bots into the same meeting), which would leave the audio graph alive and the name path silently dead — precisely the observed shape. Held as a hypothesis, not a proof: the mixed capture is created a few lines earlier in the same evaluate and ran for the whole meeting, so either the navigation preceded the evaluate or something else is at work. The fix is defensible either way — a watcher whose only job is to poll the DOM should outlive a navigation exactly as the audio hook does, and nothing about the current shape guarantees that. Capture setup is factored into a callable and re-run on every main-frame navigation. Safe by construction: each branch is guarded on its own w.__vexa* handle and a fresh document has a fresh window, so this installs one watcher per document, never two. Discrimination if it is NOT the cause: 84438cd makes a live watcher log "NO ACTIVE SPEAKER seen in Ns" every 30s. The line appearing means the watcher is running and the DOM has no active-speaker container; no line means it is not running at all. One live bot run separates them in 30 seconds. Also eliminated while looking: viewport. Xvfb runs 1920x1080, so a compact responsive layout is not the explanation.
… meeting (#852) The re-arm in 45333ed rested on a claim about Playwright that I had not tested: that a setInterval installed via page.evaluate dies at a navigation while an addInitScript one does not. Two intervals, one installed each way, then a navigation: before navigation : page.evaluate 11 ticks · addInitScript 12 ticks after navigation : page.evaluate 0 ticks · addInitScript 16 ticks So the asymmetry inside capture-bridge.ts is real: the browser bundle and the WebRTC audio hook outlive any navigation, the platform speaker watchers did not. A client that navigates after join leaves audio flowing and names dead — a transcript that is correct and attributed entirely to seg_N. This does NOT establish that Zoom navigates after the evaluate fires; the mixed capture is created a few lines earlier in the same block and ran for the whole meeting. Two possibilities remain and the blind-watcher line (84438cd) separates them in one log line on the next bot run. The point of doing it this way: the MECHANISM is deterministic and provable with a double in seconds, while only the PLATFORM's behaviour needs a live meeting. Waiting for a Zoom call to learn how Playwright handles navigation would have been the wrong trade — and it is the same split the milestone exists to make.
…ent (#854, G3) single_pass_truth.py cuts 60s chunks with 3s of overlap so a word split by a cut still appears. That shared audio is then transcribed in BOTH chunks, and the seam text lands in the reference twice. On youtube/2026-07-20-continuous-speech: reference 3870 words · 31 repeated 8-grams within 400 words of each other 122 words (3.2%) inside a duplicated region A live transcript that correctly says a phrase once cannot match a reference that says it twice, so every duplicated word scored as a miss. Recall 0.905 was understated — real streaming loss is at most ~6.3%, not the 9.5% quoted in this issue's framing, in #850's observation bundle and in the staging readiness assessment on #847. dedupe_seams joins chunks by finding, at each seam, the longest suffix of the accumulated text that also opens the next chunk, and dropping it once. Capped at 60 words — comfortably above 3s of speech — so an ordinary repeated phrase mid-chunk is never mistaken for a seam. Closes the duplication half of G3. The other half stays open and is harder: a word DROPPED at a seam is invisible without a second reference built with different cut points. Found by looking at WHICH words were missing rather than accepting how many. The largest "missing run" was the reference repeating itself — "of the moment to describe so i think we've of the moment to describe so i think we" — which is not a shape real speech takes.
…res (#854, G3) Regenerated both references with dedupe_seams and re-baselined: youtube/2026-07-20-continuous-speech recall 0.905 -> 0.920 loss 9.5% -> 8.0% jitsi/2026-07-20-capture-gaps recall 0.858 -> 0.872 loss 14.2% -> 12.8% The reference was counting its own chunk overlap as content, so a live transcript that correctly said a phrase once scored a miss against a reference that said it twice. Every loss figure quoted across #854, #850's observation bundle and the #847 staging assessment inherited that inflation. G3's duplication half closes with a measured bound. The seam-DROP half is now stated separately and stays open: a word lost at a cut is invisible without a second reference built with different cut points, so the reference's recall is unbounded from below.
…where it cannot hold (#854, G3) G3's remaining half was that a word DROPPED at a chunk cut is invisible in a single reference. It is visible in a second reference cut elsewhere, so --chunk-offset-sec builds one. fixture duty ref-vs-ref disagreement measured loss youtube/2026-07-20-continuous-speech 0.949 4.4% 8.0% jitsi/2026-07-20-capture-gaps 0.650 23.9% 12.8% On the clean control the instrument is usable — 4.4% uncertainty against an 8.0% signal — though not to the tenth of a point I have been quoting. On the gappy entry the instrument is NOISIER than the thing it measures, so recall 0.872 there cannot support any weight and is marked not citable. The cause is structural: load_session concatenates only DELIVERED frames, so every capture hole is a splice in the audio the reference is built from, and shifting the cuts lands them on different splices. A reference over holey audio is intrinsically less trustworthy — the holes are extra seams. This gives the corpus a rule it did not have: content scoring is only meaningful on high-duty entries. The threshold sits somewhere between 0.65 and 0.95; two points do not locate it. Delivery figures are untouched — duty cycle comes from the bytes, not from a model — so #850's 65.0% -> 99.9% result is unaffected by any of this.
…t the pipeline (#850, #854) Scored the same live transcript against two references over identical audio, cuts 25s apart: vs reference A (cuts 0s): matched 266 of 368 · precision 0.723 · "102 invented" vs reference B (cuts 25s): matched 314 of 368 · precision 0.853 · "54 invented" Reference B recognises 48 of the supposedly-invented words as real speech, so precision against a single reference understated by 13 points. The invention rate is at most ~14.7%, and with the two references disagreeing by 23.9% on this entry, not a number to build on. I had described that shortfall as "the sub-second-window hallucination measured as a number rather than described" — in #850's A3, its observation bundle, and this entry's own description. It was measuring the instrument. The entry's duty cycle is 0.650 and load_session concatenates only delivered frames, so every capture hole is a splice the reference reads across. CORPUS.md now says the entry's DELIVERY figure is what holds and its content columns cannot carry a claim. A3 remains partial for a corrected reason: not that fragmentation persisted, but that the evidence I cited for the invention half was my own reference's seams. Nothing delivery-side moves: duty cycle is bytes over wall time with no model in the path.
…rom duty (#854) I committed a rule from two data points — content scoring is trustworthy in proportion to duty cycle. A third point breaks it: youtube/continuous-speech duty 0.949 one speaker 4.4% ref-vs-ref youtube/witnessed-panel duty 1.000 multi-voice 7.2% jitsi/capture-gaps duty 0.650 gappy 23.9% The cleanest audio in the corpus — duty 1.000, zero gaps in 557s — is LESS stable than a 0.949 entry, and the two are the same platform and capture path. Duty is one term at most; what the audio contains looks like another, since a panel with turn-taking gives a chunk boundary more ways to go wrong than one uninterrupted speaker. CORPUS.md now says stability is measured per entry with --chunk-offset-sec and must appear beside any content figure, rather than inferred from duty. The jitsi entry stays disqualified on its own measured number, not on a rule. Also scored the witnessed panel: replay precision 0.972 against BOTH references — what the lane emits on the cleanest available audio is almost entirely real speech, and that holds across two independently-cut references. Its recall 0.311 is the unpaced-replay cadence artifact, same as the already-retracted 0.378 coverage, and is not a pipeline figure.
#854) extension-loop.mjs claimed the external loop needs no human. It does, for exactly one gesture: MIXED-lane tabs (youtube, zoom, teams) take their audio from the offscreen tabCapture stream, and chrome.tabCapture.getMediaStreamId requires the activeTab grant that ONLY a toolbar click, context menu or keyboard command produces (clients/extension/src/background.ts:538). Playwright drives pages, not browser chrome, so no driven run can produce it. autoStart still opens the session and the ingest websocket, so the failure was silent in the worst way: a tape appears, grows to 110 bytes of header, and the desktop reports capture:no-signal for the whole run. Measured on a 90s youtube run: 1 line in the tape, ch999=0f ch1000=0f, six ENGINE FAULT lines. The loop now (a) waits for the media element and reports whether currentTime advances, so a silent tab is never confused with an unminted stream, (b) nurses playback every tick and says when the player pauses itself, (c) prints the one human cue when audio is playing and no frames arrive, and (d) counts its --sec budget from the FIRST FRAME, so a run that waited two minutes for a gesture does not bank those minutes as captured data. teams-speaker.mjs joins the Teams web client with a WAV as its microphone, the sibling of zoom-speaker.mjs, reusing the selectors @vexa/join already ships so a speaker never fails for a reason the bot would not.
…eakers both (#854) Every content number in the corpus is scored against a single-pass STT reference, and that reference needs its own error bars: it duplicated its chunk overlap (122 words, 3.2%) and re-cutting it moves the answer 4.4-23.9% depending on the entry. Two "findings" this week were the reference moving, not the pipeline. A TTS clip has no such problem — the text exists before the audio does. So: build_truth_wav.py concatenates a speaker's cached clips into one microphone WAV plus a manifest of {startSec, endSec, text} per clip. Fed to a synthetic participant's fake mic, the meeting itself becomes the fixture. score_truth.py aligns the published transcript against that truth with word-level LCS (never containment — containment produced 24 false positives out of 39 here and reversed a conclusion), and because each clip owns a known wall-clock window it scores ATTRIBUTION on the same pass: of the words that survived, how many carry the name of the speaker who actually said them. Validated against four synthetic transcripts built from the truth itself, all four matching their predictions before the run: perfect recall 1.0 precision 1.0 attribution 1.0 20% words dropped recall 0.78 precision 1.0 attribution 0.999 alternate clips mislabelled recall 1.0 attribution 0.565 nothing ever named recall 1.0 attribution NONE (1470 unnamed) An unnamed segment is scored as unnamed, never as wrong: the mixed lane publishes seg_N until a hint binds a turn, and punishing that would punish honesty.
…kers through the bot (#852, #854) Two runs, two questions, and mixing them is how a green run hides a dead half: leg 1 asks whether the WORDS are right through the extension (the path a customer uses), leg 2 asks whether the NAMES are right through the bot, which is the only thing the bot uniquely carries and the only place #852 lives. Leg 2 is 90s — long enough to see hints cross the bridge, and it is not measuring content. Both legs speak the same known-truth WAVs, so neither leans on an STT reference. The speakers now print AUDIO_START=<epoch> at the instant the fake-capture file starts feeding the track. That is clip offset zero for score_truth.py: guess it and whole turns misattribute while the content numbers still look fine. The runbook states the one gesture that cannot be automated, with the citation, so no future session rediscovers it the way this one did — by watching a 90s run produce a 110-byte tape.
The first cut built one reference by interleaving both speakers' clips in absolute time. LCS is order-sensitive, so an error in the @epoch scrambled that order and the mismatch came back as lost words. Measured on a WORD-PERFECT transcript: +5s → recall 0.871, +30s → 0.607, +120s → 0.587. The speakers report AUDIO_START when the join completes, not when the first sample leaves the fake device, so a two-second slip is ordinary — this scorer would have invented double-digit streaming loss out of a clock and I would have gone looking for it in the pipeline. Scoring each speaker's own WAV against the transcript removes the clock from the content numbers entirely: order inside one speaker's truth is fixed by their own file. The epoch survives only as a coarse window filter, where seconds cannot matter. The sweep is now flat — 1.0 at +0s, +5s, +30s and +120s. That alone let two speakers claim the SAME published word (both say "the"), which cost 25 words of precision on a perfect transcript and misattributed the same 25. Speakers are now assigned disjointly, every speaker order tried, best total kept — so a perfect run scores perfectly, which is the least an instrument must do before it is allowed to grade anything else. perfect recall 1.0 precision 1.0 attribution 1.0 20% words dropped recall 0.769 precision 0.987 attribution 0.983 alternate clips mislabelled recall 1.0 attribution 0.565 nothing ever named recall 1.0 precision 1.0 attribution NONE The prior commit message claimed the epoch drives attribution and not content. That was backwards: attribution is content-aligned and barely moved (0.98-1.0 across the whole sweep), while content was the half that collapsed.
…#854, #852) speech_fixture.py already builds the mixed lane's whole input from the cached TTS clips, whose text is known before any audio exists. Driving the REAL lane over it with real STT and the real segmenter answers "is the quality good?" with nobody in the room — and the mixed lane is the one Zoom and Teams both ride. content recall 0.924 291 of 315 known words kept, in order content precision 0.936 20 published words that were never said attribution accuracy 0.947 0 wrong, 1.0% of words provisional coverage · retention 0.825 · 0.869 structure 19 segments · 0 duplicate texts · 1 hole >2s · 18 STT calls Three consecutive runs returned these to three decimals, down to the STT call count and the words returned — fixed audio through fixed cut points is deterministic even across a remote model, so this entry's tolerances are zero where the tape entries need ±15 calls. What it does NOT cover is named, so nobody reads it as more than it is: it starts at captured-signal.v1, so capture delivery is out of scope (the live legs) and so is whether the platform's DOM lit the right names (#852, the bot leg). Also: the harness printed "hints 0 matched / 8 missed (100.0% miss)" beside a 0.947 attribution accuracy with zero wrong names. That column is the hint-hop instrument — did a hint find an open turn AT THE INSTANT IT ARRIVED — and a hint 250ms into a turn lands before any audio has been transcribed into it, reports missed, and is then used correctly by the binder. Printed that way it reads exactly like a dead hint stream, which is the #852 symptom I am hunting; it now says what it measures, and score-fixture no longer calls a rise there WORSE.
Ran the bot from source against a live 35-person Zoom room (the compiled dist —
tsx injects esbuild's __name helper into page.evaluate closures and the page-side
capture dies with ReferenceError, which is a dev artifact, not production). Three
separate silences turned a diagnosable problem into "bridge-crossed=0":
1. The page-side bundle is injected with addInitScript({path}) whose .catch was
EMPTY. window.VexaBrowserUtils is the entire page side — mixed capture, gmeet
capture, every platform's speaker watcher — so a failed inject means a bot that
joins, sits there, and emits nothing at all: no audio, no hints, no page logs.
That is indistinguishable from a silent room. It now says so and names the path.
2. The page-console forwarder filtered on /perspeaker|capture|stream|vexabrowser|
audiocontext|error|fail/. Attribution lives entirely in the page-side speaker
watchers, and none of their diagnostics match those words — including the
blindness reporter written for exactly this investigation, which could never
have appeared in a bot log. `speaker` and `hint` now pass.
3. The blindness reporter drew a verdict ("selectors are stale") from one fact.
A second browser joined to the SAME meeting at the SAME moment found both
selectors present, so the verdict was wrong and it sent the investigation the
wrong way for twenty minutes. It now reports counts — [class*=speaker],
[class*=video-frame], name footers, iframes, viewport, visibility — which
separate "this client renders no speaker chrome" from "the class names moved",
and leaves the verdict to whoever reads them.
With all three fixed, from inside a genuine join (Bot admitted, 7 remote streams,
seen=1900 emitted=1900, 486.4s delivered, processor deficit 0.0s — #850's fix
holding live on Zoom): 0 [class*=speaker] nodes in the bot's DOM, and no other
WHO signal in that layout. The cause of that is still open.
) Zoom does not serve one client. A browser with hardware acceleration and a camera gets the gallery — speaker bar plus large active tile — and that is what the EXTENSION runs in, an ordinary human Chrome, where the watcher has always worked. A browser without them is served a SINGLE-TILE client, and the bot launches with --disable-gpu, --in-process-gpu and no fake capture device by design (those args cut per-bot CPU from ~4.4 cores to ~115%). Zoom even says so in a banner on the bot's own screen: "For a better meeting experience, enable the option 'Use graphics/hardware acceleration'". So the module was right, the selectors were right, and the bot was watching for chrome its client never renders. One selector was missing: `.single-main-container__video-frame`, whose name lives in the same `.video-avatar__avatar-footer` nameFromContainer already reads. Live, 26-person room, bot from source: before NO ACTIVE SPEAKER ×3 bridge-crossed=0 0 hints recorded after NO ACTIVE SPEAKER ×0 bridge-crossed=105 105 hints recorded 4 distinct speakers named, weighted as they spoke What found it was a SCREENSHOT. Every DOM probe returned zero and read like a dead page; the picture of that same instant showed the speaker's name on screen beside a live level meter. querySelectorAll can only answer questions someone thought to ask. The bot now drops a periodic PNG when VEXA_DEBUG_SHOT_DIR is set, and the blindness reporter describes whatever nodes carry a name instead of counting the ones it expected.
…reason, tallied by the replay (#868, #849) explainMatch exposes the windowMatch scan with its working shown (which gate rejected the best candidate, how many names contested, what the flicker debounce cost); resolveName narrates every provisional hold as [binder-reject]; quality-mixed tallies the reasons per run. Measured on botsig9: 26/26 rejects at the confidence gate, every one a 2-candidate contest. The 0.12.17 handoff rides along as the written ground truth.
…not silently dropped The Teams voice-outline watcher's only output is a speaker transition, so a signal it detects but cannot NAME — and a selector that has drifted to match nothing at all — both produced perfect silence: no log, a clean transcript, and attribution that reads all-provisional. That silence is what cost a live meeting to diagnose (#797). Two self-reports, no selector changes (modernizing them needs a live session): - The no-name discard at `emit` now logs that a voice signal fired but the name resolver returned empty — which name-selector strategies were tried and a sanitized element description (tag/class prefix, no user text) — rate-limited so a drifted selector reports once per window instead of every frame. - A blind-watcher interval emits `NO ACTIVE SPEAKER seen in Ns`, mirroring Zoom's #852 report word-for-word so one log grep catches both platforms, naming which of a silent room / a stale voice selector / absent tiles the DOM supports. Refs #797 #853
… installs no browser (#852) The node CI lane (gates.yml) installs no Playwright browsers, so chromium.launch() was an environment dependency, not a test: it passed locally (cached chromium) and failed in CI with 'Executable doesn't exist at .../headless_shell', exiting 1. This was the only real-browser test in the whole node lane. The test's own docstring already promised 'a DOUBLE rather than a meeting ... no flake'; the implementation just failed to deliver it. Encode Playwright's injection lifetime contract directly — addInitScript scripts re-run against every new document, a page.evaluate runs once against the current document and its timers die when that document is replaced — install one watcher each way, navigate, count which still ticks. Deterministic, browser-free, no timers.
…#797, #852) zoom-speakers.ts:233 read window.innerWidth/innerHeight unguarded inside the setInterval tick, and the tick has no try/catch: on a DOM double (or any headless context) without a window global, the blind-report branch threw 'ReferenceError: window is not defined' out of a timer callback and crashed the process — @vexa/zoom-capture#test failed in CI (uncaught, exit 1) while passing where a window happened to exist. Guard the viewport read (typeof window check, visibilityState ?? 'n/a') so the diagnostic degrades to 'n/a' instead of killing the watcher it exists to help. Align blindness.test.ts's assertion with the log text the source actually emits ('none of … exist here'). These match the versions already on the #852 stack; 797 had the pre-refinement copies.
…en hashed name-class selectors drift (#853) Live #853 instrumentation witnessed the #797 red on today's Teams web DOM: the bot IS served the participant names (visible on-screen in each tile's name pill), but every teamsNameSelector missed — the lead selector div[class*="___2u340f0"] is an extension-era minified hash that has since rotated (live tile name div now carries only atomic hashes like ___12zni01 f1cmbuwj …). extractName returned '', so the no-name guard discarded every detected voice-outline transition. Voice-outline detection itself is healthy (observing N with signal), so this is pure name-selector drift, not the #852 different-DOM class. Rather than chase a new hash that will drift again, add a selector-agnostic structural fallback that runs only after all explicit selectors miss: scan the tile's text-bearing leaves for a name-shaped string (has a letter, not a timer, not a UI-control word). The old selectors stay as the preferred fast path.
…853, #797) Hoist extractName to an exported pure fn extractTeamsSpeakerName(element, opts?) so the structural fallback can be exercised without a live meeting, and add name-drift.test.ts. The fixture is RECONSTRUCTED to mirror the live-witnessed class structure (___1504rl1 container / ___12zni01 atomic-hash leaf, no name attrs) — not a captured DOM dump. It reproduces the property that broke the resolver: a name node reachable by traversal but by no explicit selector. Asserts, offline: explicit selectors only → '' (the #797/#853 red); with the structural fallback → "Dmitry Grankin" (green); the fast path still wins when a real selector matches; and the timer/forbidden-word guard keeps a control-only tile at '' (no false name). Wired into the module's npm test; isolation stays zero-dep (hand-rolled DOM shim, no jsdom).
#853) The live Teams DOM carries mute controls as tile leaves; 'mic' does not substring-match 'mute', so a lone Mute label would have passed the name-shaped guard. Add 'mute' (covers unmute).
Contributor
Author
|
Live GREEN witnessed (see #853 comment thread): the fixed build resolved |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Live #853 instrumentation (the RED witness, 2026-07-21) caught the #797 red on today's Teams web DOM. Unlike Zoom #852 (where the bot is served no names), Teams does paint a name pill in every tile — but every
teamsNameSelectormissed. The lead selectordiv[class*="___2u340f0"]is an extension-era minified hash that has since rotated; the live name div now carries only atomic hashes like___12zni01 f1cmbuwj fv6wr3j fz5stixinside a container___1504rl1 f1euv43f ftuwxu6— nodata-tid, no aria name, matching none of the explicit selectors.extractNamereturned'', so the no-name guard discarded every voice-outline transition: perfect on-screen names, an anonymous transcript.Fix mechanism
Rather than chase a new hash that will drift again (the exact failure #797 predicted),
extractTeamsSpeakerNamegains a selector-agnostic structural fallback that runs only after every explicit selector misses: it scans the tile's text-bearing leaf nodes for a name-shaped string — has a letter, isn't a timer, isn't a UI-control/forbidden word. It reads the name the way a human reads the tile, so it survives the next hash roll. The explicit selectors stay as the preferred fast path.extractNameis hoisted out of thecreateTeamsSpeakersclosure to an exported pure function so it is testable offline; behavior for the live code path is unchanged.Observation bundle
''extractTeamsSpeakerName(tile, { structuralFallback: false }) === ''"Dmitry Grankin"extractTeamsSpeakerName(tile) === 'Dmitry Grankin'___2u340f0tile →'Legacy Name'with fallback on and off05:14+Microphonetile →''; lone00:30→''src/name-drift.test.ts— 6 checks, wired into the module'snpm test. Source-swap red→green proven: with the fallback body stubbed out the GREEN row fails (assertion failed: GREEN: the structural fallback recovers "Dmitry Grankin"); restored, all 6 pass. The fixture is RECONSTRUCTED to mirror the live-witnessed class structure (___1504rl1container /___12zni01leaf, atomic hashes only, no name attrs) — it is not a captured byte-for-byte DOM dump; what it faithfully reproduces is the property that broke the resolver (a name reachable by traversal but by no selector). Zero-dependency module preserved: hand-rolled DOM shim, no jsdom;gate:isolationgreen.npm test(teams-capture) — teams-capture + blindness + name-drift all green.node scripts/gates.mjs all→ ✅ gates green (all 33 gates).Honest scope — RED witness + fix + offline green, NOT the live green
This PR carries the live RED witness, the fix, and the offline structural regression (green). The live green — a real Teams meeting where the
dom-outlinehints resolve into named transcript rows — is still pending a clean admit and is not claimed here. It complements PR #870 (Teams de-silence), whose instrumentation is what made this name-drift diagnosis visible in the first place.Base =
main, independent (teams-capture only) — not stacked on #797/#870.Refs #853, #797.