Add device screen streaming via WebSocket#328
Draft
gabrieldonadel wants to merge 6 commits intomainfrom
Draft
Conversation
gabrieldonadel
changed the base branch from
main
to
@gabrieldonadel/add-list-devices-route
gabrieldonadel
force-pushed
the
@gabrieldonadel/add-list-devices-route
branch
from
37466d1 to
4f2ac16
Compare
gabrieldonadel
changed the base branch from
@gabrieldonadel/add-list-devices-route
to
main
Extends the Orbit local server with WebSocket support for streaming iOS simulator and Android emulator screens to a browser-based viewer. - Add StreamManager for managing capture sessions per device - Add WebSocket server on /orbit/ws for real-time frame delivery - Add /orbit/devices endpoint for listing booted devices - Add /orbit/stream endpoint serving a built-in HTML viewer - iOS capture via xcrun simctl io (JPEG frames) - Android capture via adb screenrecord (h264 stream) - Web client supports JPEG canvas rendering and WebCodecs h264 decoding https://claude.ai/code/session_01RYPF9YAuXf4XrPEg8XASRi
Adds a Swift CLI helper that captures iOS Simulator windows using ScreenCaptureKit (hardware-accelerated, 30-60fps on macOS 12.3+) with automatic fallback to CoreGraphics window capture. - Swift Package at helpers/simulator-stream/ with build.sh - ScreenCaptureKit: low-latency, GPU-accelerated frame capture - CoreGraphics fallback: CGWindowListCreateImage for older macOS - Outputs length-prefixed JPEG frames to stdout (same protocol) - StreamManager auto-detects native binary, falls back to xcrun - Build integration: macOS build.sh compiles helper, Vite bundles it https://claude.ai/code/session_01RYPF9YAuXf4XrPEg8XASRi
gabrieldonadel
force-pushed
the
claude/stream-emulator-orbit-YemlX
branch
from
201508b to
4f38c8c
Compare
- Add MJPEG capture mode for Android using adb screencap (~10-15fps) with length-prefixed PNG frames, universal browser compatibility - Improve h264 mode with auto-restart to handle screenrecord's 3-minute recording limit - Add capture mode selection (auto/mjpeg/h264) to WebSocket protocol and stream viewer UI - Add proper Annex B NAL unit parsing in the web client for h264 - Web viewer now shows active mode badge, disables h264 for iOS, handles device list format from upstream /orbit/devices endpoint - Server reports resolved captureMode back to client on stream start https://claude.ai/code/session_01RYPF9YAuXf4XrPEg8XASRi
bufferutil and utf-8-validate are optional native dependencies of the ws package. Vite fails to bundle them since they're not installed. Mark them as external so Vite skips them — ws works fine without them. https://claude.ai/code/session_01RYPF9YAuXf4XrPEg8XASRi
Simple debug page with manual WS controls, REST endpoint buttons, raw message sending, log output, and a canvas preview for frames. https://claude.ai/code/session_01RYPF9YAuXf4XrPEg8XASRi
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds real-time device screen streaming capabilities to the Electron menu bar app, allowing users to view live streams from booted iOS simulators and Android emulators through a web-based viewer.
Key Changes
New StreamManager class (
StreamManager.ts): Manages streaming sessions for multiple devicesStream viewer UI (
stream.html): New static HTML page for viewing device streamsWebSocket server integration (
LocalServer.ts):/orbit/wsfor bidirectional streaming communication/orbit/devicesendpoint to list booted simulators/emulators/orbit/streamendpoint to serve the viewer pagestart(begin streaming),stop(end streaming)Build configuration (
vite.main.config.mts): Added vite-plugin-static-copy to bundle static HTML assetsDependencies: Added
@types/wsfor WebSocket TypeScript supportImplementation Details
https://claude.ai/code/session_01RYPF9YAuXf4XrPEg8XASRi