Skip to content

fix(whisper): send timestamp_granularities as an array so Groq accepts the request (#1349) - #1511

Open
SeifEldeen-ibrahim wants to merge 1 commit into
Vexa-ai:mainfrom
SeifEldeen-ibrahim:groq-timestamp-fix
Open

SeifEldeen-ibrahim wants to merge 1 commit into
Vexa-ai:mainfrom
SeifEldeen-ibrahim:groq-timestamp-fix

Conversation

@SeifEldeen-ibrahim

@SeifEldeen-ibrahim SeifEldeen-ibrahim commented Sep 3, 2026

Copy link
Copy Markdown

Delivers issue: #1349

Contribution rights

  • Independent: I created this contribution, or otherwise have the right to submit it
    under Apache-2.0, and it is not owned or controlled by an employer, client, or other entity.
  • Employer/client authorization required: an employer, client, or other entity owns or
    may control this contribution. I am requesting Vexa's private corporate-authorization process.
  • Unsure: I need a private rights review before merge.

Every commit carries my own DCO Signed-off-by.

Observation bundle (the record of your harnessed loop)

  • C1 — reproduce without a live meeting first. ran: curl against the real Groq endpoint with
    the client's exact field name · saw: -F timestamp_granularities=word400 {"error":{"message": "unknown param \timestamp_granularities`"}}; -F 'timestamp_granularities[]=word'200` ·
    concluded: the report is accurate and the defect is in the field NAME, reproducible off-meeting.

  • C2 — establish what the bracketed form actually returns. ran: the same probe asking for
    word only, then for segment+word · saw: word-only answers "segments": null with a
    top-level words array; segment+word answers with BOTH populated · concluded: the
    issue's proposed one-line fix is incomplete.
    The client reads data.segments
    (transcription-client.ts:270), so bracketing alone yields allSegments = [], the text falls
    back to raw data.text, and every per-segment start/end/avg_logprob/no_speech_prob is
    lost — trading a loud 400 for a silent degradation. Groq's own docs corroborate: at word
    granularity the response "returns only word, start, and end timestamps". The fix must name
    segment explicitly.

  • C3 — the fix + the fixture the seam lacked. ran: npx tsx src/granularities.test.ts at base
    and at head · saw: base RED (3 of 4 checks fail, the unbracketed name is never sent — got ["word"]), head GREEN (4/4) · concluded: red→green holds on the wire bytes, mirroring the whisper client: STT model id hardcoded to "whisper-1" — OpenAI-compatible backends that validate model ids (Groq, vLLM, …) reject every live transcription; the model is not selectable per deployment #522
    model.test.ts fixture pattern for the same "validating backend" edge (D-A2).

  • C4 — live witness, my own run, before opening this. ran: bot image built from this branch,
    full compose, real Google Meet, TRANSCRIPTION_SERVICE_URL=https://api.groq.com/openai,
    TRANSCRIPTION_MODEL=whisper-large-v3-turbo, no proxy or shim anywhere in the path · saw:
    "unknown param" count 0, client faults 0, 8 speaker-attributed segments streaming
    live · concluded: the value is real end-to-end, not just green on the wire assertion.

  • C5 — negative control on the other side of the boundary. ran: the same unbracketed
    timestamp_granularities=word against a self-hosted deploy/transcription GPU unit
    (large-v3-turbo, device: cuda) · saw: HTTP 200 · concluded: the bundled unit accepts the
    field either way, which is why this never surfaced in-house and why the fix is
    behaviour-neutral for it.

Acceptance floor

#1349 is an incoming bug report rather than a prepared issue, so there is no numbered table; rows
below map to its claims.

Row Evidence
The 400 reproduces off-meeting -F timestamp_granularities=word400 unknown param; -F 'timestamp_granularities[]=word'200. Base sha 3f5c3c03.
Red→green on the wire granularities.test.ts: base RED 3/4 fail (got ["word"]), head GREEN 4/4. Negative control is the base leg itself.
Live: a Groq deployment transcribes end-to-end Full compose, Google Meet, patched bot image, direct to Groq: 0 × unknown param, 0 client faults, 8 segments. Contrast the report's 157 failures tripping stt_degraded.
No-regression for backends that ignore the field Bundled GPU unit returns 200 for both spellings (C5); response_format=verbose_json unchanged; no config flag added.
Gates gate:node — 18 package(s) build + test green. Pre-push static suite green (readme dataflow isolation isolation-py exports graph graph-py schema contract-version config-contract licenses execution-env test-isolation contract-conformance).

Exceeded the report with new witnessed value: the segments: null finding (C2). Shipping the
issue's literal one-liner would have looked like a fix and silently removed all per-segment timing
and confidence.

Docs diff (D6c)

docs/changelog.d/1349-groq-timestamp-granularities.md — one fragment, per
docs/changelog.d/README.md. Argued no-impact elsewhere: no env, API, or configuration surface
changes, so configuration.mdx / deployment.mdx need no edit — their existing Groq guidance
(TRANSCRIPTION_MODEL=whisper-large-v3-turbo) becomes true rather than changing.

Security checks (required on the diff)

  • Dependencies: none added or changed. gate:licenses green — 459 deps OSS-clean (Cat A; the
    2 pre-existing Cat-B exceptions unchanged).
  • Secrets: diff scanned; no credentials, keys, or tokens introduced (0 hits).
  • Attack surface: the change alters only outbound multipart field names on an existing
    request to an already-configured endpoint. No new network destination, no new input parsing, no
    auth path touched.
  • SAST: not run locally beyond the repo's own gate suite; CI's is authoritative.

Validation request

Preferred signer: the reporter of #1349 (their Lite + Groq rig is the natural A1 harness).

What to watch: on a Groq-pointed deployment, join one Meet call and confirm (a) no
unknown param in the bot's [TranscriptionClient] lines, (b) GET /transcripts/... non-empty,
and critically (c) segments carry real start/end values — that is the row that separates
this fix from the bracket-only version.

Deployment I validated (D12b): full Docker Compose, repo sha 3f5c3c03 + this branch, bot
image built from this checkout (make -C deploy/compose bot), published :v012 images for all
other services, long-lived clone. Env deltas from stock: TRANSCRIPTION_SERVICE_URL,
TRANSCRIPTION_SERVICE_TOKEN, TRANSCRIPTION_MODEL=whisper-large-v3-turbo.
Not validated by me: Lite, k8s/helm, hosted — honestly unclaimed.

Adjacent findings reported on the issue, deliberately NOT fixed here (kept out to hold this PR
to one defect): words still arrive empty on Groq because it returns them top-level while the
client reads them nested per segment (:276); and max_speech_duration_s (:217) /
min_silence_duration_ms (:226) are not Groq parameters either — dormant unless configured, but
the same 400 class. Happy to file either separately.

Authorship

Sole author: the human submitting this. No agent co-author trailers (D13).
Tooling disclosure (optional, welcome, never an attribution): drafted with Claude Code; every
command, probe, and live run above was executed and its raw output read before being claimed here.

…s the request (Vexa-ai#1349)

The STT client sent the multipart form part `timestamp_granularities`. The OpenAI audio API
defines that field as an ARRAY, and backends that validate the schema — Groq's
/openai/v1/audio/transcriptions — answer 400 `unknown param timestamp_granularities` to the
unbracketed name. The field is not gated by any config flag, so every live segment of a
Groq-pointed deployment failed identically until the meeting's stt_degraded breaker ended the
call.

Emit each granularity as its own `timestamp_granularities[]` part, and ask for `segment`
alongside `word`: a word-only request answers `segments: null` (Groq returns only word/start/end
at that granularity), and the client reads `data.segments` — so bracketing alone trades a loud
400 for the silent loss of every per-segment timing and confidence value.

`response_format=verbose_json`, the precondition for granularities, is already sent
unconditionally. Backends that ignore the field — the bundled deploy/transcription unit — see no
behavioural change.

granularities.test.ts is the fixture the STT seam lacked for this edge: it captures the multipart
body and asserts the bracketed name, the absence of the unbracketed one, and that both
granularities ride the wire.

Signed-off-by: SeifEldeen <seif@biami.io>
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

👋 Thanks for opening your first PR to Vexa, @SeifEldeen-ibrahim!

Highly recommended (not required): hop into our Discord
and tell us in a sentence or two what this change does and why. It helps us review
your value bundle faster and connects you with the reporter and maintainers.

Your PR is judged on its evidence — the observation bundle + the diff — not on whether you
show up. But showing up makes the whole thing smoother. See
the delivery guide for how a PR reaches merge.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

🃏 Merge card — #1511

check what it needs
Value missing state: value-signed (the value sign-off)
Diff no non-author approval on the current head sha (a new push dismisses a stale approval)

Not mergeable yet — every row above must be accepted before merge (choke point 1). Fill in what's ❌ above, then this clears automatically.

How a PR reaches merge: the merge bar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant