Support Brave Talk (iframe-embedded JaaS/8x8.vc Jitsi deployments) - #1513
Open
thiphu1012 wants to merge 2 commits into
Open
Support Brave Talk (iframe-embedded JaaS/8x8.vc Jitsi deployments)#1513thiphu1012 wants to merge 2 commits into
thiphu1012 wants to merge 2 commits into
Conversation
|
👋 Thanks for opening your first PR to Vexa, @thiphu1012! Highly recommended (not required): hop into our Discord Your PR is judged on its evidence — the observation bundle + the diff — not on whether you |
…ed deployments JaaS/8x8.vc-backed Jitsi deployments (Brave Talk and other white-labeled embeds) render the entire app — prejoin, lobby, conference, media elements — inside a cross-origin <iframe>. The top-level page has no jitsi DOM of its own, but every selector/API check in the join, admission, and capture-bridge code assumed the top frame, so the bot never found the prejoin name field, never detected admission, and (even after joining) never found <audio>/<video> elements to record/transcribe. Adds a shared frame-scanning utility (frame-utils.ts) that tries the top frame first — preserving stock meet.jit.si / self-hosted behavior exactly — then falls back to scanning child frames. Wires it into join.ts (prejoin name/join button), admission.ts (app-state, lobby, rejection/removal detection), and capture-bridge.ts (capture setup/teardown, recording tap, mic toggle). Verified live against Brave Talk: join, admission, and end-to-end transcription all confirmed working.
thiphu1012
force-pushed
the
jitsi-iframe-frame-awareness
branch
from
September 3, 2026 16:07
30263a6 to
5cdf471
Compare
Satisfies docs-current (D6c/ADR-0032) for the frame-awareness fix: adds a Brave Talk example to the Jitsi tab of Send a bot, notes that bare-URL auto-detection doesn't yet cover non-8x8.vc hosts (tracked in Vexa-ai#1512), and adds the per-PR changelog fragment.
🃏 Merge card — #1513
Not mergeable yet — every row above must be accepted before merge (choke point 1). Fill in what's ❌ above, then this clears automatically. How a PR reaches merge: the merge bar. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Brave Talk support (and any other JaaS/8x8.vc-backed white-labeled Jitsi embed) by making
the Jitsi join, admission, and capture/recording code frame-aware.
Closes #1512.
Context
JaaS/8x8.vc-backed deployments — Brave Talk (
talk.brave.com) among them — render the entireconference app (prejoin, lobby, conference, media elements) inside a cross-origin
<iframe>. Thetop-level page has no jitsi DOM of its own. Stock
meet.jit.siand most self-hosted deploymentsrender at the top level instead, and are unaffected either way.
Every DOM/API check in
join.ts,admission.ts, andcapture-bridge.tsusedpage.evaluate()/page.locator(), which only reach the top frame — so on Brave Talk the bot:"No prejoin name field detected"and looped on"waiting for admission"forever, havingnever actually entered the room).
isAdmitted()/getAppJoinedState()still couldn't seewindow.APPor the hangup button (both live in the iframe), so admission looked "unknown."startCaptureBridge/startRecordingstill targeted the topframe, so
record-chunkerfound zero<audio>/<video>elements and produced no transcriptdespite the bot visibly sitting in the call.
Fix
core/meetings/modules/join/src/jitsi/frame-utils.ts— a shared frame-scanningutility (
allJitsiFrames,findFrameWithVisibleSelector,isVisibleInAnyFrame,scanFrames,resolveConferenceFrame). Every helper tries the top frame first — byte-identical behavior forthe un-iframed case — then falls back to scanning child frames. Exported from
@vexa/join'spublic index for reuse by the bot service.
jitsi/join.ts: the prejoin name-field / Join-button lookup now usesfindFrameWithVisibleSelectorand scopes the fill + click to whichever frame it actually foundthe field in.
jitsi/admission.ts:getAppJoinedState,getLobbyState,isHangupVisible, the prejoin/conference/lobby DOM checks, and the rejection/removal text scan all now scan every frame
instead of just the top one.
capture-bridge.ts:startCaptureBridge,startRecording, and the speak path's mictoggle resolve the actual conference frame (
resolveConferenceFrame— the frame that has<audio>/<video>elements) and target it for setup, teardown, and control, only whenplatform === 'jitsi'— every other platform (and stock/self-hosted Jitsi) is untouched.page.exposeFunction()andcontext.addInitScript()already install into every frame byPlaywright's own design (confirmed in this file's existing comments), so the PCM audio bridge and
window.VexaBrowserUtilsneeded no changes — only the invocation side needed frame-targeting.Docs
docs/docs/how-to/send-a-bot.mdx— Jitsi tab now shows a Brave Talk example and notes thatbare-URL auto-detection doesn't yet cover non-8x8.vc hosts (tracked as a follow-up in Jitsi bot can't join or transcribe iframe-embedded deployments (Brave Talk / JaaS/8x8.vc) #1512).
docs/changelog.d/1513-jitsi-iframe-frame-awareness.md— per-PR changelog fragment (docs-current/ ADR-0032).
Test case
Live end-to-end verification against a real Brave Talk room (
talk.brave.com), self-hostedcompose stack,
make botbuild of this branch:"No prejoin name field detected — proceeding to admission checks", loops"Still waiting for admission"until the escalation VNC debugview confirms the browser is still sitting on the pre-join screen with the name field visibly
pre-filled (
"PT's notetaker") — the bot just never saw it."Name entered"→"Join clicked"→"Bot immediately admitted (no lobby)". Confirmed via the meeting API (status: "active")."[record-chunker] 1 media elements (strict)"→"connected element 1/1"→"combined 1 streams"→"MediaRecorder started". PollingGET /transcripts/jitsi/{room}returns real transcribed segments matching what was spokeninto the call, e.g.:
{ "text": "Hello, this meeting will have a discussion about the chop head...", "language": "en", "completed": true }admission.test.ts,join.test.ts) pass unchanged —the test
pagestub was extended withframes()/mainFrame()returning itself, soallJitsiFrames()degenerates to exactly the prior single-frame behavior for these tests.npx tsx src/jitsi/admission.test.ts→ 6/6 pass;npx tsx src/jitsi/join.test.ts→ 11/11pass.
make botimage build (TypeScript compile across the whole@vexa/botbuild scope)succeeds clean.
Deployment validated: self-hosted Compose (
make all+make bot), macOS host via OrbStack,fresh clone of
mainat6d6d38a2, self-hosted CPU transcription unit (small→mediummodel). Not yet validated on Lite or k8s/Helm.
Known follow-up (not in scope here)
password.ts's room-password dialog handling is still top-frame-only — an iframe-embedded,password-protected Jitsi room would need the same treatment. Not hit by Brave Talk (no password
prompt in this flow).
meeting_urlplatform auto-detection (resolvePlatform) doesn't recognizetalk.brave.comor other non-8x8.vcJaaS custom domains —platformmust be sent explicitlyfor now (documented above).