fix(whisper): send timestamp_granularities as an array so Groq accepts the request (#1349) - #1511
Open
SeifEldeen-ibrahim wants to merge 1 commit into
Open
SeifEldeen-ibrahim wants to merge 1 commit into
SeifEldeen-ibrahim wants to merge 1 commit into
Conversation
…s the request (Vexa-ai#1349) The STT client sent the multipart form part `timestamp_granularities`. The OpenAI audio API defines that field as an ARRAY, and backends that validate the schema — Groq's /openai/v1/audio/transcriptions — answer 400 `unknown param timestamp_granularities` to the unbracketed name. The field is not gated by any config flag, so every live segment of a Groq-pointed deployment failed identically until the meeting's stt_degraded breaker ended the call. Emit each granularity as its own `timestamp_granularities[]` part, and ask for `segment` alongside `word`: a word-only request answers `segments: null` (Groq returns only word/start/end at that granularity), and the client reads `data.segments` — so bracketing alone trades a loud 400 for the silent loss of every per-segment timing and confidence value. `response_format=verbose_json`, the precondition for granularities, is already sent unconditionally. Backends that ignore the field — the bundled deploy/transcription unit — see no behavioural change. granularities.test.ts is the fixture the STT seam lacked for this edge: it captures the multipart body and asserts the bracketed name, the absence of the unbracketed one, and that both granularities ride the wire. Signed-off-by: SeifEldeen <seif@biami.io>
|
👋 Thanks for opening your first PR to Vexa, @SeifEldeen-ibrahim! Highly recommended (not required): hop into our Discord Your PR is judged on its evidence — the observation bundle + the diff — not on whether you |
🃏 Merge card — #1511
Not mergeable yet — every row above must be accepted before merge (choke point 1). Fill in what's ❌ above, then this clears automatically. How a PR reaches merge: the merge bar. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Delivers issue: #1349
Contribution rights
under Apache-2.0, and it is not owned or controlled by an employer, client, or other entity.
may control this contribution. I am requesting Vexa's private corporate-authorization process.
Every commit carries my own DCO
Signed-off-by.Observation bundle (the record of your harnessed loop)
C1 — reproduce without a live meeting first. ran:
curlagainst the real Groq endpoint withthe client's exact field name · saw:
-F timestamp_granularities=word→400 {"error":{"message": "unknown param \timestamp_granularities`"}};-F 'timestamp_granularities[]=word'→200` ·concluded: the report is accurate and the defect is in the field NAME, reproducible off-meeting.
C2 — establish what the bracketed form actually returns. ran: the same probe asking for
wordonly, then forsegment+word· saw: word-only answers"segments": nullwith atop-level
wordsarray;segment+wordanswers with BOTH populated · concluded: theissue's proposed one-line fix is incomplete. The client reads
data.segments(
transcription-client.ts:270), so bracketing alone yieldsallSegments = [], the text fallsback to raw
data.text, and every per-segmentstart/end/avg_logprob/no_speech_probislost — trading a loud 400 for a silent degradation. Groq's own docs corroborate: at word
granularity the response "returns only word, start, and end timestamps". The fix must name
segmentexplicitly.C3 — the fix + the fixture the seam lacked. ran:
npx tsx src/granularities.test.tsat baseand at head · saw: base RED (3 of 4 checks fail,
the unbracketed name is never sent — got ["word"]), head GREEN (4/4) · concluded: red→green holds on the wire bytes, mirroring the whisper client: STT model id hardcoded to "whisper-1" — OpenAI-compatible backends that validate model ids (Groq, vLLM, …) reject every live transcription; the model is not selectable per deployment #522model.test.tsfixture pattern for the same "validating backend" edge (D-A2).C4 — live witness, my own run, before opening this. ran: bot image built from this branch,
full compose, real Google Meet,
TRANSCRIPTION_SERVICE_URL=https://api.groq.com/openai,TRANSCRIPTION_MODEL=whisper-large-v3-turbo, no proxy or shim anywhere in the path · saw:"unknown param"count 0, client faults 0, 8 speaker-attributed segments streaminglive · concluded: the value is real end-to-end, not just green on the wire assertion.
C5 — negative control on the other side of the boundary. ran: the same unbracketed
timestamp_granularities=wordagainst a self-hosteddeploy/transcriptionGPU unit(
large-v3-turbo,device: cuda) · saw: HTTP 200 · concluded: the bundled unit accepts thefield either way, which is why this never surfaced in-house and why the fix is
behaviour-neutral for it.
Acceptance floor
#1349 is an incoming bug report rather than a prepared issue, so there is no numbered table; rows
below map to its claims.
-F timestamp_granularities=word→400 unknown param;-F 'timestamp_granularities[]=word'→200. Base sha3f5c3c03.granularities.test.ts: base RED 3/4 fail (got ["word"]), head GREEN 4/4. Negative control is the base leg itself.unknown param, 0 client faults, 8 segments. Contrast the report's 157 failures trippingstt_degraded.response_format=verbose_jsonunchanged; no config flag added.gate:node— 18 package(s) build + test green. Pre-push static suite green (readme dataflow isolation isolation-py exports graph graph-py schema contract-version config-contract licenses execution-env test-isolation contract-conformance).Exceeded the report with new witnessed value: the
segments: nullfinding (C2). Shipping theissue's literal one-liner would have looked like a fix and silently removed all per-segment timing
and confidence.
Docs diff (D6c)
docs/changelog.d/1349-groq-timestamp-granularities.md— one fragment, perdocs/changelog.d/README.md. Argued no-impact elsewhere: no env, API, or configuration surfacechanges, so
configuration.mdx/deployment.mdxneed no edit — their existing Groq guidance(
TRANSCRIPTION_MODEL=whisper-large-v3-turbo) becomes true rather than changing.Security checks (required on the diff)
gate:licensesgreen — 459 deps OSS-clean (Cat A; the2 pre-existing Cat-B exceptions unchanged).
request to an already-configured endpoint. No new network destination, no new input parsing, no
auth path touched.
Validation request
Preferred signer: the reporter of #1349 (their Lite + Groq rig is the natural A1 harness).
What to watch: on a Groq-pointed deployment, join one Meet call and confirm (a) no
unknown paramin the bot's[TranscriptionClient]lines, (b)GET /transcripts/...non-empty,and critically (c) segments carry real
start/endvalues — that is the row that separatesthis fix from the bracket-only version.
Deployment I validated (D12b): full Docker Compose, repo sha
3f5c3c03+ this branch, botimage built from this checkout (
make -C deploy/compose bot), published:v012images for allother services, long-lived clone. Env deltas from stock:
TRANSCRIPTION_SERVICE_URL,TRANSCRIPTION_SERVICE_TOKEN,TRANSCRIPTION_MODEL=whisper-large-v3-turbo.Not validated by me: Lite, k8s/helm, hosted — honestly unclaimed.
Adjacent findings reported on the issue, deliberately NOT fixed here (kept out to hold this PR
to one defect):
wordsstill arrive empty on Groq because it returns them top-level while theclient reads them nested per segment (
:276); andmax_speech_duration_s(:217) /min_silence_duration_ms(:226) are not Groq parameters either — dormant unless configured, butthe same 400 class. Happy to file either separately.
Authorship
Sole author: the human submitting this. No agent co-author trailers (D13).
Tooling disclosure (optional, welcome, never an attribution): drafted with Claude Code; every
command, probe, and live run above was executed and its raw output read before being claimed here.