refactor(test-skills): restructure test skills as self-contained architecture

Summary:
Replace the dual-purpose v1/v2 skill structure with self-contained skills
that work both standalone and as orchestrated sub-agents.

Individual skills (9):
- Remove Phase 1-4 outer structure and BEGIN/END SUB-AGENT markers
- Add Step 1-5 structure (install, start server, verify, test, cleanup)
- Use MCPCALL shorthand with dynamic port discovery
- Add recovery/retry guidance for transient failures
- Remove Follower suite from test-ui (not present in poke example)

Orchestrator (test-all):
- Orchestrator starts dev servers (sub-agents can't run background processes)
- Dynamic port discovery via .mcp.json files
- Sub-agents read skill files directly, start from Step 3
- Wait for completion notifications instead of polling
- No improvised bash commands — only pre-approved node scripts

New scripts:
- scripts/test-prep.mjs: clone/install/cleanup example directories
- scripts/test-servers.mjs: start/ports/stop dev server lifecycle
- All orchestrator bash operations converted to node scripts (zero permission prompts)

Settings: add allowlist entries for pnpm, npx, npm run fresh:install

Reviewed By: zjm-meta

Differential Revision: D95235621

fbshipit-source-id: 345a359881f045ec60f5b612fd97a9f81b898e6b

Author: felixtrz

.claude/settings.json

@@ -35,11 +35,15 @@
       "mcp__iwsdk-dev-mcp__ecs_diff",
       "Bash(npm run dev *)",
       "Bash(npm run fresh:dev *)",
+      "Bash(npm run fresh:install *)",
+      "Bash(pnpm install)",
+      "Bash(pnpm build:tgz)",
+      "Bash(pnpm -v)",
       "Bash(lsof *)",
       "Bash(kill *)",
       "Bash(sleep *)",
-      "Bash(claude *)",
-      "Bash(unset *)"
+      "Bash(node *)",
+      "Bash(npx *)"
     ]
   },
   "enableAllProjectMcpServers": true,

.claude/skills/test-all/SKILL.md

@@ -1,143 +1,206 @@
 ---
 name: test-audio
-description: Automated test for the audio system (AudioSource loading, playback state, stop). Tests ECS audio state transitions using IWER MCP tools against the audio example. Dynamic entity discovery. Run from examples/audio/ with the dev server running. Note that actual audio output cannot be verified — only ECS state.
-argument-hint: [--suite loading|playback|stop|all]
+description: "Test audio system (AudioSource loading, playback state, stop, spatial audio) against the audio example using mcp-call.mjs WebSocket CLI."
+argument-hint: "[--suite loading|playback|stop|all]"
 ---
 
 # Audio System Test
 
-Automated test suite for verifying AudioSource loading, playback state transitions, and AudioSystem behavior using IWER MCP tools.
+Run 6 test suites covering audio loading, playback trigger, stop, system registration, component schema, and stability.
 
-**Target Example:** `examples/audio`
+All tool calls go through `scripts/mcp-call.mjs` via WebSocket — no MCP server, no permission prompts.
 
-Run this skill from the `examples/audio/` directory with the dev server running (`npm run dev`).
+**Configuration:**
+- EXAMPLE_DIR: /Users/felixz/Projects/immersive-web-sdk/examples/audio
+- ROOT: /Users/felixz/Projects/immersive-web-sdk
 
-**Important**: Actual audio output cannot be verified via MCP tools. These tests verify ECS state transitions only (loading, play requested, is playing, stop).
-
-## Pre-test Setup
+**SHORTHAND**: Throughout this document, `MCPCALL` means:
+```
+node /Users/felixz/Projects/immersive-web-sdk/scripts/mcp-call.mjs --port <PORT>
+```
+where `<PORT>` is the port number discovered in Step 2.
 
+**Tool calling pattern**: Every tool call is a Bash command using the MCPCALL shorthand:
 ```
-mcp__iwsdk-dev-mcp__browser_reload_page
-mcp__iwsdk-dev-mcp__xr_accept_session
-mcp__iwsdk-dev-mcp__browser_get_console_logs(count: 20, level: ["error"]) → no errors (audio autoplay warnings are acceptable)
+MCPCALL --tool <TOOL_NAME> --args '<JSON_ARGS>' 2>/dev/null
 ```
 
----
+- `<TOOL_NAME>` uses MCP-style names (e.g. `browser_reload_page`, `xr_accept_session`). The script handles translation internally.
+- `<JSON_ARGS>` is a JSON object string. Omit `--args` if no arguments needed.
+- Output is JSON on stdout. Parse it to check assertions.
+- Use `--timeout 20000` for operations that may take longer (reload, accept_session, screenshot).
+- Always append `2>/dev/null` to suppress TLS warnings.
 
-## Test Suites
+**IMPORTANT**: Run each Bash command one at a time. Parse the JSON output and verify assertions before moving to the next command. Do NOT chain multiple mcp-call commands together.
 
-### Suite 1: Audio Loading
+**IMPORTANT**: When the instructions say "wait N seconds", use `sleep N` as a separate Bash command.
+
+**IMPORTANT**: Boolean values in `ecs_set_component` must be actual JSON booleans (`value: true`), NOT strings (`value: "true"`). Strings silently fail to coerce.
 
-**What we're testing**: AudioSource loads its buffer automatically.
+---
 
-#### Test 1.1: Find Audio Entity
+## Step 1: Install Dependencies
 
+```bash
+cd /Users/felixz/Projects/immersive-web-sdk/examples/audio && npm run fresh:install
 ```
-mcp__iwsdk-dev-mcp__ecs_find_entities(withComponents: ["AudioSource"])
-→ At least 1 entity. Save as <audio>.
+
+Wait for this to complete before proceeding.
+
+---
+
+## Step 2: Start Dev Server
+
+Start the dev server as a background task using the Bash tool's `run_in_background: true` parameter:
+
+```bash
+cd /Users/felixz/Projects/immersive-web-sdk/examples/audio && npm run dev
 ```
 
-The audio example uses a GLXF level that creates entities via composition. The Spinner entity has an AudioSource.
+**IMPORTANT**: This command MUST be run with `run_in_background: true` on the Bash tool — do NOT append `&` to the command itself.
+
+Once the background task is launched, poll the output for Vite's ready message (up to 60s). Read the task output or use `tail` to watch for a line containing `Local:`. The output will contain a URL like `https://localhost:5173/`. Extract the port number from this URL and save it as `<PORT>`. All subsequent `MCPCALL` commands use this port.
 
-#### Test 1.2: Verify Loaded State
+If the server fails to start within 60 seconds, report FAIL for all suites and skip to Step 5.
 
+---
+
+## Step 3: Verify Connectivity
+
+```bash
+MCPCALL --tool ecs_list_systems 2>/dev/null
 ```
-mcp__iwsdk-dev-mcp__ecs_query_entity(entityIndex: <audio>, components: ["AudioSource"])
+
+This must return JSON with a list of systems. If it fails:
+1. Check the dev server output for errors
+2. Try killing and restarting the server (Step 2)
+3. If it still fails, report FAIL for all suites and skip to Step 5
+
+---
+
+## Step 4: Run Test Suites
+
+### Pre-test Setup
+
+Run these commands in order:
+
+1. `MCPCALL --tool browser_reload_page --timeout 20000 2>/dev/null`
+   Then: `sleep 3`
+
+2. `MCPCALL --tool xr_accept_session --timeout 20000 2>/dev/null`
+   Then: `sleep 2`
+
+3. `MCPCALL --tool browser_get_console_logs --args '{"count":20,"level":["error"]}' 2>/dev/null`
+   Assert: No error-level logs. Audio autoplay warnings are acceptable.
+
+---
+
+### Suite 1: Audio Loading
+
+**Test 1.1: Find Audio Entity**
+```bash
+MCPCALL --tool ecs_find_entities --args '{"withComponents":["AudioSource"]}' 2>/dev/null
 ```
+Assert: At least 1 entity. Save the first as `<audio>`.
+
+The audio example uses a GLXF level that creates entities via composition. The Spinner entity has an AudioSource.
 
-**Assert**:
+**Test 1.2: Verify Loaded State**
+```bash
+MCPCALL --tool ecs_query_entity --args '{"entityIndex":<audio>,"components":["AudioSource"]}' 2>/dev/null
+```
+Assert:
 - `src` contains an audio file path (e.g., `.mp3`)
 - `_loaded` = `true` (buffer loaded)
 - `_loading` = `false` (not currently loading)
 - `_isPlaying` = `false` (not playing yet — unless autoplay is set)
 - `volume` = `1`
 - `positional` = `true`
 
-#### Test 1.3: Pool Created
-
-**Assert**: `_pool` exists with `available` array matching `maxInstances`.
+**Test 1.3: Pool Created**
+Assert: `_pool` exists with `available` array matching `maxInstances`.
 
 ---
 
 ### Suite 2: Playback Trigger
 
-**What we're testing**: Setting `_playRequested: true` triggers the AudioSystem to play.
-
-#### Test 2.1: Request Play
-
+**Test 2.1: Request Play**
+```bash
+MCPCALL --tool ecs_set_component --args '{"entityIndex":<audio>,"componentId":"AudioSource","field":"_playRequested","value":true}' 2>/dev/null
 ```
-mcp__iwsdk-dev-mcp__ecs_set_component(entityIndex: <audio>, componentId: "AudioSource",
-  field: "_playRequested", value: "true")
-```
-
-**Assert**: `_playRequested` was consumed (response shows `newValue: false` — the AudioSystem processed it within the same frame).
+Assert: `_playRequested` was consumed (response shows `newValue: false` — the AudioSystem processed it within the same frame).
 
-#### Test 2.2: Play with Loop for Observable State
+**Test 2.2: Play with Loop for Observable State**
 
-To observe `_isPlaying: true`, set `loop: true` first, then request play:
+Set `loop: true` first, then request play:
+```bash
+MCPCALL --tool ecs_set_component --args '{"entityIndex":<audio>,"componentId":"AudioSource","field":"loop","value":true}' 2>/dev/null
 ```
-mcp__iwsdk-dev-mcp__ecs_set_component(entityIndex: <audio>, componentId: "AudioSource",
-  field: "loop", value: "true")
-mcp__iwsdk-dev-mcp__ecs_set_component(entityIndex: <audio>, componentId: "AudioSource",
-  field: "_playRequested", value: "true")
+```bash
+MCPCALL --tool ecs_set_component --args '{"entityIndex":<audio>,"componentId":"AudioSource","field":"_playRequested","value":true}' 2>/dev/null
 ```
 
 Then query:
+```bash
+MCPCALL --tool ecs_query_entity --args '{"entityIndex":<audio>,"components":["AudioSource"]}' 2>/dev/null
 ```
-mcp__iwsdk-dev-mcp__ecs_query_entity(entityIndex: <audio>, components: ["AudioSource"])
-→ _isPlaying: true (looping sound keeps playing)
-```
+Assert: `_isPlaying` = `true` (looping sound keeps playing).
 
 ---
 
 ### Suite 3: Stop
 
-#### Test 3.1: Request Stop
-
+**Test 3.1: Request Stop**
+```bash
+MCPCALL --tool ecs_set_component --args '{"entityIndex":<audio>,"componentId":"AudioSource","field":"_stopRequested","value":true}' 2>/dev/null
 ```
-mcp__iwsdk-dev-mcp__ecs_set_component(entityIndex: <audio>, componentId: "AudioSource",
-  field: "_stopRequested", value: "true")
-```
-
-**Assert**: `_stopRequested` consumed, `_isPlaying` becomes `false`.
+Assert: `_stopRequested` consumed, `_isPlaying` becomes `false`.
 
 ---
 
 ### Suite 4: System Registration
 
+```bash
+MCPCALL --tool ecs_list_systems 2>/dev/null
 ```
-mcp__iwsdk-dev-mcp__ecs_list_systems
-→ AudioSystem at priority 0
-→ Config keys: enableDistanceCulling, cullingDistanceMultiplier
-→ audioEntities: ≥ 1
-```
+Assert:
+- AudioSystem at priority 0
+- Config keys: `enableDistanceCulling`, `cullingDistanceMultiplier`
+- `audioEntities` >= 1
 
 ---
 
 ### Suite 5: Component Schema
 
+```bash
+MCPCALL --tool ecs_list_components 2>/dev/null
 ```
-mcp__iwsdk-dev-mcp__ecs_list_components
-→ AudioSource fields:
-  Core: src (FilePath), volume (Float32), loop (Boolean), autoplay (Boolean)
-  Spatial: positional (Boolean), refDistance, rolloffFactor, maxDistance, distanceModel, coneInnerAngle, coneOuterAngle, coneOuterGain
-  Behavior: playbackMode (Enum), maxInstances (Int8), crossfadeDuration (Float32), instanceStealPolicy (Enum)
-  Control: _playRequested, _pauseRequested, _stopRequested (Boolean), _fadeIn, _fadeOut (Float32)
-  State: _pool (Object), _instances (Object), _isPlaying (Boolean), _buffer (Object), _loaded, _loading (Boolean)
-```
+Assert AudioSource fields:
+- Core: `src` (FilePath), `volume` (Float32), `loop` (Boolean), `autoplay` (Boolean)
+- Spatial: `positional` (Boolean), `refDistance`, `rolloffFactor`, `maxDistance`, `distanceModel`, `coneInnerAngle`, `coneOuterAngle`, `coneOuterGain`
+- Behavior: `playbackMode` (Enum), `maxInstances` (Int8), `crossfadeDuration` (Float32), `instanceStealPolicy` (Enum)
+- Control: `_playRequested`, `_pauseRequested`, `_stopRequested` (Boolean), `_fadeIn`, `_fadeOut` (Float32)
+- State: `_pool` (Object), `_instances` (Object), `_isPlaying` (Boolean), `_buffer` (Object), `_loaded`, `_loading` (Boolean)
 
 ---
 
 ### Suite 6: Stability
 
+```bash
+MCPCALL --tool browser_get_console_logs --args '{"count":30,"level":["error","warn"]}' 2>/dev/null
 ```
-mcp__iwsdk-dev-mcp__browser_get_console_logs(count: 30, level: ["error", "warn"])
-→ No errors. Audio autoplay warnings are acceptable.
-```
+Assert: No application-level errors. Audio autoplay warnings and pre-existing 404 resource errors from page load are acceptable.
 
 ---
 
-## Results Summary
+## Step 5: Cleanup & Results
+
+Kill the dev server:
+```bash
+kill $(lsof -t -i :<PORT>) 2>/dev/null
+```
+
+Output a summary table:
 
 ```
 | Suite                    | Result    |
@@ -150,6 +213,20 @@ mcp__iwsdk-dev-mcp__browser_get_console_logs(count: 30, level: ["error", "warn"]
 | 6. Stability             | PASS/FAIL |
 ```
 
+If any suite fails, include which assertion failed and actual vs expected values.
+
+---
+
+## Recovery
+
+If at any point a transient error occurs (server crash, WebSocket timeout, connection refused, etc.) that is NOT caused by a source code bug:
+1. Kill the dev server: `kill $(lsof -t -i :<PORT>) 2>/dev/null`
+2. Restart: re-run Step 2 to start a fresh dev server (port may change)
+3. Re-run the Pre-test Setup (reload, accept session)
+4. Retry the failed suite
+
+Only give up after one retry attempt per suite. If the same suite fails twice, mark it FAIL and continue to the next suite.
+
 ---
 
 ## Known Issues & Workarounds
@@ -167,25 +244,10 @@ If `_stopRequested` and `_playRequested` are set simultaneously, stop wins.
 IWER runs in a browser context where the AudioContext may be suspended until a user gesture. The MCP tools can verify ECS state transitions but cannot confirm actual audio output.
 
 ### Audio example uses GLXF level
-The audio example loads entities from `./glxf/Composition.glxf`. Entities are not created in index.js — they come from the GLXF composition. The SpinSystem applies rotation to entities with AudioSource. Use `ecs_find_entities` to discover them dynamically.
+The audio example loads entities from `./glxf/Composition.glxf`. Entities are not created in index.js — they come from the GLXF composition. Use `ecs_find_entities` to discover them dynamically.
 
-## Architecture Notes
-
-### Playback State Machine
-```
-[AudioSource added] → _loading=true → loadAudio() → _loaded=true, _pool created
-   ↓ (_playRequested=true)
-handlePlaybackRequests():
-  Restart: stop all → create new instance → _isPlaying=true
-  Overlap: add instance (steal if full) → _isPlaying=true
-  Ignore: skip if already playing
-  FadeRestart: fade out current → fade in new → _isPlaying=true
-   ↓ (onended or _stopRequested)
-releaseInstance() → if no instances left → _isPlaying=false
-```
+### Boolean values must be JSON booleans
+When setting boolean fields (like `_playRequested`, `loop`, `_stopRequested`) via `ecs_set_component`, the `value` must be a JSON boolean (`true`), not a string (`"true"`). Strings silently fail.
 
-### PlaybackMode Enum
-- `restart` — Stop current, play new
-- `overlap` — Add concurrent instance (up to maxInstances)
-- `ignore` — Skip if already playing
-- `fade-restart` — Crossfade from current to new
+### Entity indices change on reload
+Never cache entity indices across page reloads. Always re-discover via `ecs_find_entities`.