[WIP] VideoCallClient platform abstraction

[darioalessandro]

_This PR introduces cross-platform support for videocall-client, allowing it to be used by both WASM (browser) and native (desktop/server) applications.

It refactors bot and videocall-cli to depend on the new NativeVideoCallClient API, eliminating significant code duplication for WebTransport, heartbeat logic, and VP9 encoding. This consolidates core communication and encoding logic into videocall-client and videocall-codecs, making videocall-client a unified library for all client types and improving overall maintainability.

Type of Change

• New feature
• Enhancement

Testing

• Tests written or updated

---

 

[cursor]

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
_{Learn more about Cursor Agents}

[darioalessandro]

@testradav this PR will take a while but it adds true cross-platform to VideoCallClient

[darioalessandro]

this looks like slop

[darioalessandro]

Now I have the full picture. Let me map the layers.

Current Architecture: Two Completely Separate Stacks

WASM stack (5 layers, ~1200 lines):
ConnectionController → ConnectionManager → Connection → Task → WebMedia trait
timers multi-server RTT heartbeat enum WS|WT send_bytes()
gloo::Interval election, state AES encrypt fallback web_sys APIs
Rc Callback<>, Rc Rc

Native stack (1 file, 389 lines):
NativeVideoCallClient (monolithic)
tokio::spawn, mpsc, Arc
heartbeat, send loop, inbound consumer all inline

Why They Can't Simply Merge Today

The root blocker is the concurrency model mismatch:

┌───────────────┬────────────────────────────────────────────────┬───────────────────────────────┐
│ │ WASM │ Native │
├───────────────┼────────────────────────────────────────────────┼───────────────────────────────┤
│ Smart pointer │ Rc<Cell<>>, Rc<RefCell<>> │ Arc, Arc<Mutex<>> │
├───────────────┼────────────────────────────────────────────────┼───────────────────────────────┤
│ Callbacks │ videocall_types::Callback (Rc-based, not Send) │ Box<dyn Fn() + Send + Sync> │
├───────────────┼────────────────────────────────────────────────┼───────────────────────────────┤
│ Timers │ gloo::Interval │ tokio::time::interval │
├───────────────┼────────────────────────────────────────────────┼───────────────────────────────┤
│ Spawn │ wasm_bindgen_futures::spawn_local │ tokio::spawn │
├───────────────┼────────────────────────────────────────────────┼───────────────────────────────┤
│ Timestamps │ js_sys::Date::now() │ std::time::SystemTime │
└───────────────┴────────────────────────────────────────────────┴───────────────────────────────┘

The Callback type permeates the entire WASM connection stack — ConnectOptions, ConnectionManagerOptions, Connection,
all use it. It's !Send, so it can't be used in tokio tasks. This isn't a superficial difference; it's baked into every
struct.

What IS Duplicated (and can be shared today)

Despite the concurrency differences, there's real protocol logic duplicated:

1. Heartbeat packet construction — connection.rs:117-141 and native_client.rs:298-319 build identical
   MediaPacket{HEARTBEAT} → PacketWrapper structs
2. Connection packet construction — native_client.rs:257-277 (not done in connection.rs because Task does it, but same
   logic)
3. RTT packet construction — connection_manager.rs:372-388
4. State enum — ConnectionState could be shared
5. Protocol constants — heartbeat interval (1s), election period, etc.

Proposed Plan: Incremental Unification

Phase 1: Shared protocol module (easy, do now)

Extract pure packet construction and state types into a cross-platform module:

// videocall-client/src/protocol.rs — always compiled, no platform deps

pub fn build_heartbeat_packet(
userid: &str, timestamp: f64,
video: bool, audio: bool, screen: bool,
aes: Option<&Aes128State>,
) -> PacketWrapper { ... }

pub fn build_connection_packet(userid: &str, meeting_id: &str) -> Result { ... }

pub fn build_rtt_packet(userid: &str, timestamp: f64, aes: &Aes128State) -> Result { ... }

pub enum ConnectionState { Testing {...}, Connected {...}, Reconnecting {...}, Failed {...} }

pub const HEARTBEAT_INTERVAL_MS: u32 = 1000;
pub const RTT_PROBE_INTERVAL_MS: u32 = 200;

Both WASM Connection and native NativeVideoCallClient call these instead of building packets inline. This eliminates
the most error-prone duplication (packet format divergence).

Phase 2: Transport trait (moderate, do next)

The WebMedia trait already exists but is WASM-only. Generalize it:

// videocall-transport/src/transport_trait.rs — cross-platform

pub trait Transport: Send {
fn send_bytes(&self, data: Vec) -> Result<()>;
fn is_connected(&self) -> bool;
}

// WASM: impl Transport for WebSocketTask, WebTransportTask
// Native: impl Transport for NativeWebTransportSession, NativeWebSocketClient

This lets the Connection struct become generic over transport.

Phase 3: Unified Connection (larger refactor)

Once transports have a shared trait, the Connection struct (heartbeat + state tracking + send) can become
cross-platform:

pub struct Connection<T: Transport> {
transport: T,
heartbeat_handle: platform::IntervalHandle, // already cross-platform!
state: Arc, // works on both
// ...
}

The platform module already provides cross-platform IntervalHandle, now_ms(), and spawn() — so timers/timestamps are
solved.

The big change: Replace Callback<> with mpsc channels for events. This is needed because:

• Channels are Send (work in native tokio tasks)
• They work fine in WASM too (tokio mpsc compiles to wasm32)
• They're more composable (callers can select! over events)

This makes ConnectionManager (election, RTT, multi-server) reusable on native too.

What I'd recommend

Phase 1 is low-risk, high-value — do it in this PR. It prevents packet format divergence between the two stacks without
touching the concurrency model.

Phase 2+3 is the real unification but it means refactoring the WASM client to drop Callback<> in favor of channels,
which touches yew-ui integration. That's a separate PR.

Want me to start with Phase 1 — extracting the shared protocol module and wiring both stacks to use it?