v1.3.0: PSP video decode, remote automation, network recovery, docs overhaul#100
v1.3.0: PSP video decode, remote automation, network recovery, docs overhaul#100AndrewAltimit merged 66 commits intomainfrom
Conversation
Performance optimizations (P1-P7): - P1: Pre-allocated video texture double-buffer (eliminates ~30MB/s alloc churn) - P2: Remove redundant StreamFrame copies — send raw AVCC directly, clone SPS/PPS only on keyframes (eliminates ~3MB/s alloc churn) - P3: Pre-allocated DecodedFrame RGBA double-buffer via static FRAME_BUFFERS with decode_into() (eliminates ~15MB/s alloc churn) - P4: Semaphore-based wakeup for VIDEO_STREAM_QUEUE (replaces 5ms polling) - P5: Audio thread immediate re-pop after output_blocking (eliminates ~20 wasted wakeups per AAC frame) - P6: Range-based D-cache flush in AAC decoder init - P7: Conditional vlog suppression during active decode Streaming resilience: - Reduced TLS socket read timeout from 30s to 10s for faster stall detection - Auto-reconnect with HTTP Range request on TLS connection drop (up to 3 retries) - I/O thread heartbeat logging for download progress diagnosis ME decode investigation: - Added DECODE_STEP atomic watchdog (tracks which sceMpeg call is executing) - Main thread watchdog detects ME hangs and logs step number - Confirmed: sceMpegAvcDecode (step 2) deadlocks after ~80-95 consecutive frames on 656x480 Main profile content — D-cache, DPB flush, decoder recreation all ruled out as mitigations - Periodic "ME rest" skip-to-keyframe as partial workaround Note: psp dependency temporarily uses local path for decode_into + flush + DECODE_STEP additions (rust-psp branch: feat/decode-into) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sceMpegAvcDecodeFlush, sceMpegFlushAllStream, and sceMpegInit all cause hard freezes when called during active H.264 decoding with mpeg_vsh370.prx. Remove all mid-stream flush attempts until a safe reset mechanism is found. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When sceMpegAvcDecode deadlocks (step=2, ~90 frames at 656x480), the watchdog now switches to audio-only mode instead of leaving the video frozen. Clears the preview texture and lets audio continue. Investigation findings: - SPS ref frame patching (3→1) made hang WORSE (1 frame vs 90), proving DPB size is NOT the cause - sceMpegAvcDecodeFlush crashes mid-stream (hard freeze) - sceMpegFlushAllStream crashes mid-stream (hard freeze) - sceMpegInit mid-stream crashes (hard freeze) - Internal semaphore at mpeg_data+0x66c is the mutex lock, not the ME completion wait — signalling it doesn't unblock AvcDecode - The hang is in ME hardware register polling inside mpeg_vsh370.prx, after the mutex is acquired - frame_width (768 vs 512) to sceMpegAvcDecode doesn't affect the hang Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reverse-engineered the full sceMpegAvcDecode call chain in mpeg_vsh370.prx:
PRX dispatch (0x71c0) → real AvcDecode (0x1078) → ME trigger (0x8650)
→ kernel import stub (0x9fd4) → WaitEventFlag(infinite) → DEADLOCK
At runtime, after loading mpeg_vsh370.prx:
1. Compute PRX base from resolved import stub address
2. Locate the `jal 0x9fd4` instruction at PRX VA 0x8678
3. After 75 decoded frames, patch it to `addiu $v0, $zero, -1`
(return error, skip ME kernel call)
4. Video thread continues without deadlock, audio plays normally,
last decoded frame stays on screen
This gives ~2.5s of H.264 video decode at 656x480 before the patch
activates, with continuous audio playback and no crashes/freezes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Combines runtime PRX patching (skip ME kernel call after 85 frames) with 33ms decode throttle for ~2.8s of visible video per stream. System is fully stable — tested 940+ frames with no crash/deadlock. Also attempted SceMediaEngineRpc event flag approach to unblock stuck decodes, but the event flag is kernel-only (not visible from user mode). Next step: use oasis-plugin-psp (kernel PRX) to find and signal the event flag with a timeout. Investigation finding: event flag scan of UIDs 1-4096 found no "SceMediaEngineRpc" — confirms it's a kernel-mode event flag created by sceMeCodecWrapper, inaccessible from user space. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Added me_watchdog module to oasis-plugin-psp (kernel PRX) that attempts to find SceMediaEngineRpc event flag and hook WaitEventFlag with timeout. Finding: sceKernelReferEventFlagStatus returns errors for ALL UIDs from kernel plugin threads — event flags are not enumerable from this context. The delayed scan (5s intervals for 60s) also failed after sceMpegInit. The runtime PRX patching in the EBOOT (at 85 frames) remains the working solution for preventing ME deadlocks on >480p content. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New modules for the standard sceMpeg ringbuffer H.264 decode path: psmf.rs — MPEG-PS pack builder: - Generates 2048-byte PSMF header with stream descriptors - Generates first pack with system header + private_stream_2 index - Wraps H.264 Annex B data in correctly formatted MPEG-PS packs - Each pack exactly 2048 bytes (matching ringbuffer packet size) - Proper MPEG-2 SCR timestamps and PES headers with PTS - Format derived from byte-level analysis of real Sony PMF file psmf_decode.rs — Ringbuffer decoder: - Uses standard sceMpeg mode 0 (not mode 4/5) - Uses AvMpegBase system module (not mpeg_vsh370.prx) - sceMpegRingbufferPut → kernel MPEG-PS demux → sceMpegGetAvcAu - Different ME firmware code path that may not have 90-frame bug - Skeleton with TODO for callback integration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fully functional PsmfDecoder with: - Global callback state for kernel-invoked ringbuffer callback - Proper sceMpegRingbufferConstruct with callback function - PSMF header validation via sceMpegQueryStreamOffset - Flat pack buffer with D-cache flush for kernel visibility - feed_and_decode() API: wraps Annex B → MPEG-PS → ringbuffer → decode - Comprehensive logging for on-device debugging Uses mode 0 (standard) with AvMpegBase — completely different ME firmware code path from the NAL direct path (mode 5) that deadlocks. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
play_stream() now tries PsmfDecoder first (mode 0, AvMpegBase). Falls back to NalDecoder (mode 5, mpeg_vsh370.prx) if PSMF init fails. Added avcc_to_annex_b_simple() helper for PSMF path which needs H.264 data in Annex B format (start codes instead of length prefixes). PSMF path: StreamFrame → AVCC→Annex B → MPEG-PS packs → ringbuffer NAL path: StreamFrame → raw AVCC → sceMpegGetAvcNalAu (existing) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New journal entry documenting the full investigation into the PSP Media Engine firmware deadlock in sceMpegAvcDecode at ~90 frames for >480p content. Covers: - P1-P7 streaming performance optimizations (48MB/s alloc churn eliminated) - Systematic deadlock hunting (12+ approaches tested and ruled out) - Reverse engineering mpeg_vsh370.prx call chain via Capstone - Runtime binary patching of loaded PRX modules from user-mode Rust - Event flag investigation dead end - PSMF ringbuffer alternative path (byte-level PMF format analysis) Also updates article 07 nav link and journal index. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sceUsb functions are user-mode exports not available from kernel PRX import stubs. Use sctrlHENFindFunction to resolve sceUsbStart, sceUsbActivate, sceUsbDeactivate, sceUsbGetState at runtime from the kernel USB driver module. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AlwaysUSB (CFW+ v1.7) handles USB storage auto-activation on XMB. Our devloop just polls for command files and handles launches/reboots. Removes static mut USB function pointers that caused nightly errors. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Devloop resolves all sceNet/sceNetInet/sceNetApctl functions at runtime via sctrlHENFindFunction (kernel PRX import stubs don't link to user-mode net libraries). Connects to WiFi profile 1 without dialog, starts TCP server on port 9293 for remote commands. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New minimal kernel PRX (153KB) dedicated to remote dev automation: - WiFi auto-connect to saved profile 1 (no dialog) - TCP server on port 9293 for commands - Commands: ping, screenshot, reboot, launch <path> - All net functions resolved via NID (kernel PRX compatible) - Small net pool (16KB) to avoid OOM on boot - 4KB thread stack, no heap allocator - Separate from oasis-plugin-psp to avoid overlay/hook conflicts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Kernel PRXs built with rust-psp crash in XMB/VSH context. Devloop now runs in game context only — waits for the EBOOT to initialize the network stack (WiFi dialog), then opens TCP server on port 9293. Net functions resolved via NID from sceNetInet_Library/sceNetApctl_Library which become available after the EBOOT loads net modules. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause of PRX crashes: writing to ms0: during AlwaysUSB init (race condition). Fixed with 10s delay — kernel can write to ms0: even while USB storage is active after init completes. TCP server waits for EBOOT to init network, then binds port 9293. Commands: ping, screenshot, reboot, launch <path> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
On XMB, the devloop PRX waits 20 seconds (for host to deploy EBOOT via AlwaysUSB), then auto-launches ms0:/PSP/GAME/OASISOS/EBOOT.PBP. Host can prevent auto-launch by creating ms0:/seplugins/devloop_noauto. Full automation cycle: PSP boots → AlwaysUSB → host deploys → PRX auto-launches → WiFi TCP server on :9293 → host sends commands → reboot → repeat. Only human action: power on the PSP. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Main thread polls devloop_cmd.txt (works when not on USB storage). Worker thread starts TCP server on :9293 after EBOOT inits network. Both paths call shared execute_cmd() for: reboot, screenshot, launch. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New cmd_server module in oasis-backend-psp (user-mode, not kernel PRX): - Spawns thread after boot, waits for WiFi connection - TCP server on port 9293 - Commands: ping, screenshot, reboot, log - Uses EBOOT's existing network stack (no NID resolution needed) - screenshot saves raw VRAM to ms0:/seplugins/devloop_screen.raw - log command sends last 2KB of eboot.log over TCP - reboot exits to XMB via sceKernelExitGame Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Auto-connect to saved WiFi profile 1 on background thread at boot (no dialog needed). Falls back to dialog if auto-connect fails. - 'exit' command: returns to XMB (sceKernelExitGame) - 'reboot' command: restarts EBOOT (sceKernelLoadExec) - Network init moved to cmd_server background thread to avoid UI freeze Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete hands-free PSP development cycle verified: reboot → cold restart → AutoStart → OASIS OS → WiFi auto-connect → TCP :9293 ready → pong (30 seconds, zero human intervention) Commands: ping, log, screenshot, exit (XMB), reboot (cold reset) Cold reboot via scePowerRequestColdReset (NID 0x0442D852 from ARK-4). WiFi auto-connects to saved profile 1 on background thread. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New 'deploy <size>' command receives raw EBOOT binary over TCP, writes to temp file, then renames over the live EBOOT. Combined with 'reboot' (cold reset + AutoStart), this enables fully remote build-deploy-test cycles with zero USB interaction. Protocol: echo 'deploy <size>' + cat EBOOT.PBP | nc <ip> 9293 Script: ./scripts/psp-devloop.sh cycle <eboot_path> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…cted - cmd_server: 5s WLAN hardware warmup delay, apctl state checks instead of psp::net::is_connected(), WLAN switch detection, error code logging, retry logic in server_main, 16KB stack for format! allocations - network: ensure_net_init early-returns when apctl already has IP, skipping redundant module loading, net init, and WiFi dialog - helpers: skip reinit_gu_frame when WiFi already connected (no dialog was shown), preventing GU command buffer corruption on TV Guide launch Full zero-touch dev cycle verified: build → TCP deploy → cold reboot → AutoStart → WiFi auto-connect → TCP :9293 ready (~40s, no human input). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TCP command server now supports full remote UI automation: - press <button>: inject button press/release (cross, circle, d-pad, triggers, start, select, triangle, square) - hold <button> <ms>: hold a button for N milliseconds - cursor <x> <y>: move cursor to absolute position - status: JSON with kiosk app, free memory, max block, frame count - screencap: stream raw 480x272 ABGR framebuffer over TCP - logfull: last 8KB of eboot.log (vs 2KB for log) Input injection uses a lock-free SpscQueue between the TCP server thread and the main loop. Injected CursorMove events update the backend's internal cursor position for correct hit-testing. Main loop reports kiosk state + heap stats to cmd_server every 15 frames via update_status(). psp-devloop.sh updated with all new commands including screencap-to-PNG conversion via ffmpeg. Verified: remotely launched TV Guide by cursor-clicking its icon. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously the TV Guide window was blank in desktop/windowed mode because render_desktop.rs had no handler for the "tvguide" window ID. Added draw_tvguide_windowed() which renders the channel list with numbers, names, episode counts, and selection highlight directly into the WM window content area. Shows "Now playing" when tuned. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace SDI-based channel list in kiosk mode with the same draw_tvguide_windowed() used in windowed mode, scaled to fill the full content area. Both views now render identically. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
draw_tvguide_windowed() now handles all TV Guide states: - Channel list with selection highlight and episode counts - Playing audio: centered green status + episode title - Downloading: progress bar + percentage + episode title - Error: red error message Both fullscreen kiosk and windowed desktop mode use this single renderer, adapting layout to the given (x,y,w,h) bounds. Replaces separate views::draw_tv_playing() and views_sdi::update_tv_channels paths in kiosk mode. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New standalone crate: oasis-recovery-psp — a minimal PSP EBOOT that provides a WiFi TCP file server for remote EBOOT/PRX replacement. Replaces ARK-4's default recovery menu (triggered by holding R-trigger during boot). When the main OASIS OS EBOOT is bricked, this recovery binary still loads and accepts file uploads over WiFi. Features: - WiFi auto-connect to saved profiles (no dialog) - TCP server on port 9293 (same protocol as cmd_server) - upload: write any file to ms0: over WiFi - reboot: cold hardware reset - status: free memory, WiFi state - 154KB total size Deploy: cp EBOOT.PBP ms0:/PSP/SAVEDATA/ARK_01234/RECOVERY.PBP With two actuators (power slider + R-trigger), this enables a fully closed loop where the PSP can always be recovered remotely regardless of what code is deployed to the main EBOOT. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New section documenting the network recovery EBOOT: - 154KB Rust EBOOT replaces ARK-4 recovery menu - WiFi auto-connect + TCP file server in recovery mode - Verified: uploaded 4.8MB EBOOT via recovery, rebooted to OASIS - Two-actuator design makes PSP unbrickable over WiFi - Updated automation loop from 6 to 7 layers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New "The PSP Story" section leading with the most impressive work: TLS 1.3, ME hardware video decode, kernel overlay, in-memory streaming, closed-loop remote development, network recovery EBOOT - Links to Developer's Journal for technical deep dives - Restructured Key Features: browser engine first, PSP-specific section - Condensed crate table from individual rows to layer-based grouping - Added PSP Remote Development section with devloop.sh examples - Added recovery EBOOT to PSP build instructions - Tighter architecture description, same diagram - Developer's Journal added to documentation list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CI pipeline (main-ci.yml): - Build PSP plugin PRX and recovery EBOOT alongside main EBOOT - Collect all PSP artifacts with descriptive names: OASIS-OS.PBP, oasis-overlay.prx, RECOVERY.PBP - Updated release notes with PSP install paths and quick start - Fixed SDL2 → SDL3 in release notes Version bump: 1.2.0 → 1.3.0 across workspace, PSP backend, PSP plugin, and recovery EBOOT. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace ASCII art with Mermaid graph that renders natively on GitHub. Dark theme styling with accent colors, PSP backend highlighted. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…cleaner layout Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… PSIX OASIS_OS was NOT ported from a 2006 C homebrew. It was built from scratch in Rust starting February 2026, inspired by PSP homebrew shells like PSIX. Updated all references across: - README.md, CLAUDE.md, AGENTS.md - docs/design.md, docs/adr/005-toml-skin-system.md - site/index.html, site/journal/01-unified-embedded-os.html - crates/oasis-app/src/vfs_setup.rs (VFS about page) - crates/oasis-backend-wasm/src/vfs_content.rs (WASM about page) - crates/oasis-app-settings/src/lib.rs (settings about text) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…discovery Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ders Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Skin rename (breaking: OASIS_SKIN=cyberpunk → OASIS_SKIN=balatro): - skins/cyberpunk/ → skins/balatro/ - screenshots/cyberpunk/ → screenshots/balatro/ - All references updated across source, docs, site New screenshots for protanopia and tritanopia accessibility skins. Fixed missing pub use for protanopia_skin/tritanopia_skin in mod.rs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…skins Skin rename (breaking: OASIS_SKIN=cyberpunk → OASIS_SKIN=balatro): - skins/cyberpunk/ → skins/balatro/ (shader wallpaper matches Balatro) - screenshots/cyberpunk/ → screenshots/balatro/ - All references updated across source, docs, site Removed protanopia and tritanopia skins (too similar to classic). Theme::protanopia()/tritanopia() utility functions kept in oasis-ui. Skin count remains 18. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixed the ASAN test failure: strings.toml title was still "OASIS Cyberpunk", and TOML comments/constants had stale references. All 185 oasis-skin tests pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Gemini AI Code ReviewIssues
Previous Issues(none) Suggestions
Notes
Generated by Gemini AI (gemini-3.1-pro-preview). Supplementary to human reviews. |
Automated fix by Claude in response to Gemini/Codex review. Iteration: 1/5 Co-Authored-By: AI Review Agent <noreply@anthropic.com>
Review Response Agent (Iteration 1)Status: Changes committed, pushing... Commit: Fixed Issues
Ignored Issues
Deferred to Human
Notes
Automated summary of agent fixes. |
Gemini AI Incremental ReviewThis is an incremental review focusing on changes since the last review. Issues
Previous Issues
Suggestions(none) Notes
Generated by Gemini AI (gemini-3.1-pro-preview). Supplementary to human reviews. |
Automated fix by Claude in response to Gemini/Codex review. Iteration: 2/5 Co-Authored-By: AI Review Agent <noreply@anthropic.com>
Review Response Agent (Iteration 2)Status: Changes committed, pushing... Commit: Fixed Issues
Ignored Issues
Deferred to Human
Notes
Automated summary of agent fixes. |
Gemini AI Incremental ReviewThis is an incremental review focusing on changes since the last review. Issues
Previous Issues
Suggestions
Notes
Generated by Gemini AI (gemini-3.1-pro-preview). Supplementary to human reviews. |
Automated fix by Claude in response to Gemini/Codex review. Iteration: 3/5 Co-Authored-By: AI Review Agent <noreply@anthropic.com>
Review Response Agent (Iteration 3)Status: Changes committed, pushing... Commit: Fixed Issues
Ignored Issues
Deferred to Human
Notes
Automated summary of agent fixes. |
Gemini AI Incremental ReviewThis is an incremental review focusing on changes since the last review. Issues (if any)(none) Previous Issues (for incremental reviews)
Suggestions (if any)(none) Notes
Generated by Gemini AI (gemini-3.1-pro-preview). Supplementary to human reviews. |
Review Response Agent (Iteration 4)Status: No changes needed
Fixed Issues
Ignored Issues
Deferred to Human
Notes
The agent reviewed feedback but no file modifications were detected. |
Summary
Major release covering PSP Media Engine video decode, closed-loop remote development infrastructure, network recovery, and comprehensive documentation updates.
PSP Video Streaming
sceKernelWaitEventFlagwith 5s timeout for ME deadlock recoveryRemote Development Automation
upload <size> <path>) for remote PRX updatesscripts/psp-devloop.shhelper script for all commandsNetwork Recovery EBOOT
ms0:/PSP/SAVEDATA/ARK_01234/RECOVERY.PBPTV Guide UI
Documentation
Skin Changes
CI/Release
Test plan