Skip to content

Support Brave Talk (iframe-embedded JaaS/8x8.vc Jitsi deployments) - #1513

Open
thiphu1012 wants to merge 2 commits into
Vexa-ai:mainfrom
thiphu1012:jitsi-iframe-frame-awareness
Open

Support Brave Talk (iframe-embedded JaaS/8x8.vc Jitsi deployments)#1513
thiphu1012 wants to merge 2 commits into
Vexa-ai:mainfrom
thiphu1012:jitsi-iframe-frame-awareness

Conversation

@thiphu1012

@thiphu1012 thiphu1012 commented Sep 3, 2026

Copy link
Copy Markdown

Summary

Adds Brave Talk support (and any other JaaS/8x8.vc-backed white-labeled Jitsi embed) by making
the Jitsi join, admission, and capture/recording code frame-aware.

Closes #1512.

Context

JaaS/8x8.vc-backed deployments — Brave Talk (talk.brave.com) among them — render the entire
conference app (prejoin, lobby, conference, media elements) inside a cross-origin <iframe>. The
top-level page has no jitsi DOM of its own. Stock meet.jit.si and most self-hosted deployments
render at the top level instead, and are unaffected either way.

Every DOM/API check in join.ts, admission.ts, and capture-bridge.ts used
page.evaluate() / page.locator(), which only reach the top frame — so on Brave Talk the bot:

  1. Never found the prejoin name field or Join button (logged a false
    "No prejoin name field detected" and looped on "waiting for admission" forever, having
    never actually entered the room).
  2. Once join itself was patched, isAdmitted()/getAppJoinedState() still couldn't see
    window.APP or the hangup button (both live in the iframe), so admission looked "unknown."
  3. Once admission was patched too, startCaptureBridge/startRecording still targeted the top
    frame, so record-chunker found zero <audio>/<video> elements and produced no transcript
    despite the bot visibly sitting in the call.

Fix

  • New: core/meetings/modules/join/src/jitsi/frame-utils.ts — a shared frame-scanning
    utility (allJitsiFrames, findFrameWithVisibleSelector, isVisibleInAnyFrame, scanFrames,
    resolveConferenceFrame). Every helper tries the top frame first — byte-identical behavior for
    the un-iframed case — then falls back to scanning child frames. Exported from @vexa/join's
    public index for reuse by the bot service.
  • jitsi/join.ts: the prejoin name-field / Join-button lookup now uses
    findFrameWithVisibleSelector and scopes the fill + click to whichever frame it actually found
    the field in.
  • jitsi/admission.ts: getAppJoinedState, getLobbyState, isHangupVisible, the prejoin/
    conference/lobby DOM checks, and the rejection/removal text scan all now scan every frame
    instead of just the top one.
  • capture-bridge.ts: startCaptureBridge, startRecording, and the speak path's mic
    toggle resolve the actual conference frame (resolveConferenceFrame — the frame that has
    <audio>/<video> elements) and target it for setup, teardown, and control, only when
    platform === 'jitsi'
    — every other platform (and stock/self-hosted Jitsi) is untouched.

page.exposeFunction() and context.addInitScript() already install into every frame by
Playwright's own design (confirmed in this file's existing comments), so the PCM audio bridge and
window.VexaBrowserUtils needed no changes — only the invocation side needed frame-targeting.

Docs

Test case

Live end-to-end verification against a real Brave Talk room (talk.brave.com), self-hosted
compose stack, make bot build of this branch:

  1. Before fix: bot navigates to the room, logs "No prejoin name field detected — proceeding to admission checks", loops "Still waiting for admission" until the escalation VNC debug
    view confirms the browser is still sitting on the pre-join screen with the name field visibly
    pre-filled ("PT's notetaker") — the bot just never saw it.
  2. After join+admission fix: bot logs "Name entered""Join clicked"
    "Bot immediately admitted (no lobby)". Confirmed via the meeting API (status: "active").
  3. After capture fix: page console logs "[record-chunker] 1 media elements (strict)"
    "connected element 1/1""combined 1 streams""MediaRecorder started". Polling
    GET /transcripts/jitsi/{room} returns real transcribed segments matching what was spoken
    into the call, e.g.:
    { "text": "Hello, this meeting will have a discussion about the chop head...",
      "language": "en", "completed": true }
  4. Regression check: existing unit tests (admission.test.ts, join.test.ts) pass unchanged —
    the test page stub was extended with frames()/mainFrame() returning itself, so
    allJitsiFrames() degenerates to exactly the prior single-frame behavior for these tests.
    npx tsx src/jitsi/admission.test.ts → 6/6 pass; npx tsx src/jitsi/join.test.ts → 11/11
    pass.
  5. Full make bot image build (TypeScript compile across the whole @vexa/bot build scope)
    succeeds clean.

Deployment validated: self-hosted Compose (make all + make bot), macOS host via OrbStack,
fresh clone of main at 6d6d38a2, self-hosted CPU transcription unit (smallmedium
model). Not yet validated on Lite or k8s/Helm.

Known follow-up (not in scope here)

  • password.ts's room-password dialog handling is still top-frame-only — an iframe-embedded,
    password-protected Jitsi room would need the same treatment. Not hit by Brave Talk (no password
    prompt in this flow).
  • Bare-meeting_url platform auto-detection (resolvePlatform) doesn't recognize
    talk.brave.com or other non-8x8.vc JaaS custom domains — platform must be sent explicitly
    for now (documented above).

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

👋 Thanks for opening your first PR to Vexa, @thiphu1012!

Highly recommended (not required): hop into our Discord
and tell us in a sentence or two what this change does and why. It helps us review
your value bundle faster and connects you with the reporter and maintainers.

Your PR is judged on its evidence — the observation bundle + the diff — not on whether you
show up. But showing up makes the whole thing smoother. See
the delivery guide for how a PR reaches merge.

…ed deployments

JaaS/8x8.vc-backed Jitsi deployments (Brave Talk and other white-labeled embeds)
render the entire app — prejoin, lobby, conference, media elements — inside a
cross-origin <iframe>. The top-level page has no jitsi DOM of its own, but every
selector/API check in the join, admission, and capture-bridge code assumed the
top frame, so the bot never found the prejoin name field, never detected
admission, and (even after joining) never found <audio>/<video> elements to
record/transcribe.

Adds a shared frame-scanning utility (frame-utils.ts) that tries the top frame
first — preserving stock meet.jit.si / self-hosted behavior exactly — then
falls back to scanning child frames. Wires it into join.ts (prejoin name/join
button), admission.ts (app-state, lobby, rejection/removal detection), and
capture-bridge.ts (capture setup/teardown, recording tap, mic toggle).

Verified live against Brave Talk: join, admission, and end-to-end transcription
all confirmed working.
@thiphu1012
thiphu1012 force-pushed the jitsi-iframe-frame-awareness branch from 30263a6 to 5cdf471 Compare September 3, 2026 16:07
Satisfies docs-current (D6c/ADR-0032) for the frame-awareness fix: adds a
Brave Talk example to the Jitsi tab of Send a bot, notes that bare-URL
auto-detection doesn't yet cover non-8x8.vc hosts (tracked in Vexa-ai#1512), and
adds the per-PR changelog fragment.
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

🃏 Merge card — #1513

check what it needs
Value missing state: value-signed (the value sign-off)
Diff no non-author approval on the current head sha (a new push dismisses a stale approval)
Acceptance every acceptance leg delivered on #1512

Not mergeable yet — every row above must be accepted before merge (choke point 1). Fill in what's ❌ above, then this clears automatically.

How a PR reaches merge: the merge bar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Jitsi bot can't join or transcribe iframe-embedded deployments (Brave Talk / JaaS/8x8.vc)

1 participant