Integrate ARToolKit WASM detector with ImageBitmap→OffscreenCanvas→detector pipeline

README.md

@@ -1,24 +1,260 @@
-# arjs-plugin-artoolkit
-```markdown
 # @ar-js-org/arjs-plugin-artoolkit
 
-Minimal ARToolKit detection plugin scaffold for AR.js core.
+ARToolKit marker detection plugin for AR.js core with WebAssembly support.
+
+## Features
+
+- **Web Worker-based detection** - Runs marker detection in a separate thread for better performance
+- **ImageBitmap support** - Zero-copy frame transfer from main thread to worker (browser)
+- **Cross-platform** - Works in browsers (Worker) and Node.js (worker_threads)
+- **ARToolKit WASM integration** - Supports simple square marker detection
+- **Event-driven API** - Emits marker lifecycle events (found/updated/lost)
+
+## Installation
+
+```bash
+npm install @ar-js-org/arjs-plugin-artoolkit
+```
+
+## ARToolKit WASM Setup
+
+The plugin requires ARToolKit WASM binaries for real marker detection. Without them, the plugin runs in stub mode.
+
+### Option 1: Using artoolkit5-js (Recommended)
+
+```bash
+npm install artoolkit5-js
+```
+
+The loader will automatically detect and use the WASM module.
+
+### Option 2: Manual WASM Binary
+
+Place the `artoolkit.wasm` file in `src/worker/artoolkit/`:
+
+```
+src/worker/artoolkit/
+  ├── loader.js
+  ├── artoolkit.wasm  # <- Place WASM binary here
+  └── README.md
+```
+
+See [src/worker/artoolkit/README.md](src/worker/artoolkit/README.md) for more details.
+
+### Fallback Mode
+
+If WASM is not available:
+- The plugin will initialize successfully with console warnings
+- Detection will return empty results
+- All other functionality works normally
 
 ## Usage
 
+### Basic Integration
+
 Register with the Engine plugin manager:
 
 ```js
 import { ArtoolkitPlugin } from '@ar-js-org/arjs-plugin-artoolkit';
 
-engine.pluginManager.register('artoolkit', new ArtoolkitPlugin({ /* options */ }));
+const plugin = new ArtoolkitPlugin({
+  worker: true,              // Enable worker-based detection (default: true)
+  lostThreshold: 5,          // Frames before marker considered lost
+  frameDurationMs: 100       // Expected frame duration in ms
+});
+
+engine.pluginManager.register('artoolkit', plugin);
 await engine.pluginManager.enable('artoolkit');
 ```
 
+### Event Handling
+
 The plugin emits events on the engine event bus:
-- `ar:markerFound`
-- `ar:markerUpdated`
-- `ar:markerLost`
 
-This repo contains a skeleton. Detection and worker/WASM integration will be implemented in follow-up work.
-```
+```js
+// Marker first detected
+engine.eventBus.on('ar:markerFound', (data) => {
+  console.log('Marker found:', data.id);
+  console.log('Pose matrix:', data.poseMatrix); // Float32Array[16]
+  console.log('Confidence:', data.confidence);  // 0.0 - 1.0
+  console.log('Corners:', data.corners);        // [{x, y}, ...]
+});
+
+// Marker updated (tracking)
+engine.eventBus.on('ar:markerUpdated', (data) => {
+  // Same data structure as markerFound
+});
+
+// Marker lost (not detected for lostThreshold frames)
+engine.eventBus.on('ar:markerLost', (data) => {
+  console.log('Marker lost:', data.id);
+});
+
+// Worker ready
+engine.eventBus.on('ar:workerReady', () => {
+  console.log('Detection worker initialized');
+});
+
+// Worker errors
+engine.eventBus.on('ar:workerError', (error) => {
+  console.error('Worker error:', error);
+});
+```
+
+### Sending Frames to Plugin
+
+The plugin subscribes to `engine:update` events and expects frames with ImageBitmap:
+
+```js
+// Capture from video element
+const video = document.getElementById('video');
+const imageBitmap = await createImageBitmap(video);
+
+// Send to plugin via engine update
+engine.eventBus.emit('engine:update', {
+  id: frameId,
+  timestamp: Date.now(),
+  imageBitmap: imageBitmap,  // Transferred to worker (zero-copy)
+  width: imageBitmap.width,
+  height: imageBitmap.height
+});
+```
+
+**Important:** The ImageBitmap is transferred to the worker and cannot be used after the emit. The worker will automatically close it after processing.
+
+## Examples
+
+### Simple Marker Detection
+
+A complete webcam-based marker detection example is available in `examples/simple-marker/`:
+
+```bash
+# Start a local server (requires Node.js)
+npx http-server examples/simple-marker -p 8080
+
+# Or use Python
+python3 -m http.server 8080 --directory examples/simple-marker
+```
+
+Then open http://localhost:8080 in your browser.
+
+The example demonstrates:
+- Webcam capture with getUserMedia
+- ImageBitmap creation from video frames
+- Frame submission to the plugin
+- Event handling and console output
+- Status monitoring
+
+See [examples/simple-marker/script.js](examples/simple-marker/script.js) for the implementation.
+
+## Development
+
+### Install Dependencies
+
+```bash
+npm install
+```
+
+### Run Tests
+
+```bash
+npm test
+```
+
+### Build
+
+```bash
+npm run build
+```
+
+## Architecture
+
+### Detection Pipeline (Browser)
+
+```
+Video/Camera
+  ↓ createImageBitmap()
+ImageBitmap
+  ↓ postMessage (transfer)
+Worker: OffscreenCanvas
+  ↓ drawImage()
+Worker: ImageData
+  ↓ ARToolKit WASM
+Worker: Detections
+  ↓ postMessage
+Plugin: Event Emission
+```
+
+### Worker Implementation
+
+- **Platform-agnostic**: Supports both browser Worker and Node worker_threads
+- **OffscreenCanvas**: Used for ImageBitmap → ImageData conversion
+- **Resource management**: Automatically closes ImageBitmap after processing
+- **Defensive**: Falls back gracefully if WASM is not available
+
+### Plugin Features
+
+- **Marker lifecycle tracking**: Maintains state for found/updated/lost transitions
+- **Configurable thresholds**: Adjustable lost detection timing
+- **Zero-copy transfer**: Uses ImageBitmap transferables in browsers
+- **Cross-platform**: Works in both browser and Node.js environments
+
+## API Reference
+
+### ArtoolkitPlugin
+
+#### Constructor Options
+
+```typescript
+{
+  worker?: boolean;           // Enable worker (default: true)
+  lostThreshold?: number;     // Frames before lost (default: 5)
+  frameDurationMs?: number;   // Frame duration (default: 200ms)
+  sweepIntervalMs?: number;   // Lost marker check interval (default: 100ms)
+}
+```
+
+#### Methods
+
+- `async init(core)` - Initialize plugin with engine core
+- `async enable()` - Enable plugin and start worker
+- `async disable()` - Disable plugin and stop worker
+- `dispose()` - Alias for disable()
+- `getMarkerState(markerId)` - Get current state of a marker
+
+#### Events Emitted
+
+- `ar:workerReady` - Worker initialized and ready
+- `ar:markerFound` - New marker detected
+- `ar:markerUpdated` - Tracked marker updated
+- `ar:markerLost` - Marker no longer detected
+- `ar:workerError` - Worker error occurred
+
+## Troubleshooting
+
+### No detections
+
+1. Check browser console for WASM loading warnings
+2. Verify WASM binary is in correct location (see Setup)
+3. Ensure camera feed is active and markers are visible
+4. Check marker quality and lighting conditions
+
+### Worker not starting
+
+1. Verify browser supports Web Workers and ES modules
+2. Check for CSP (Content Security Policy) restrictions
+3. Ensure script is served over HTTPS or localhost
+
+### Performance issues
+
+1. Reduce frame rate (increase timeout in example)
+2. Lower camera resolution
+3. Check CPU usage - detection is compute-intensive
+
+## License
+
+MIT
+
+## Contributing
+
+Contributions welcome! Please follow the existing code style and add tests for new features.

examples/simple-marker/README.md

@@ -0,0 +1,116 @@
+# Simple Marker Example
+
+This example demonstrates how to load and track a pattern marker using the ARToolKit plugin.
+
+## Setup Instructions
+
+### 1. Install Dependencies
+
+From the repository root, install the dependencies:
+
+```bash
+npm install
+```
+
+### 2. Serve the Example
+
+You need to serve the example through a local web server because the Worker and ES modules require proper MIME types. You can use any static file server. Here are a few options:
+
+#### Option A: Using Python (if installed)
+
+```bash
+# From repository root
+python3 -m http.server 8080
+```
+
+Then open: http://localhost:8080/examples/simple-marker/index.html
+
+#### Option B: Using Node.js http-server
+
+```bash
+# Install http-server globally if not already installed
+npm install -g http-server
+
+# From repository root
+http-server -p 8080
+```
+
+Then open: http://localhost:8080/examples/simple-marker/index.html
+
+#### Option C: Using VS Code Live Server
+
+If you're using VS Code with the Live Server extension:
+1. Right-click on `index.html`
+2. Select "Open with Live Server"
+
+### 3. Using the Example
+
+1. Once the page loads, you'll see "Initializing..." status
+2. Wait for the worker to be ready (status will change to "Worker ready")
+3. Click the "Load Marker" button to load the Hiro pattern marker
+4. Watch the event log to see the marker loading process
+
+## What's Happening
+
+The example demonstrates:
+
+1. **Plugin Initialization**: Creating and initializing the ArtoolkitPlugin
+2. **Worker Communication**: The plugin starts a Web Worker for marker detection
+3. **Pattern Loading**: When you click "Load Marker", it calls `plugin.loadMarker()` with:
+   - Pattern URL: `/examples/simple-marker/data/patt.hiro`
+   - Marker size: `1` (world units)
+4. **Event Handling**: The page listens for AR events:
+   - `ar:workerReady` - Worker is initialized and ready
+   - `ar:markerFound` - A marker is detected for the first time
+   - `ar:markerUpdated` - Marker position/orientation updated
+   - `ar:markerLost` - Marker is no longer visible
+
+## Pattern File
+
+The `data/patt.hiro` file is a placeholder pattern file. In a real implementation:
+
+1. You would use actual ARToolKit pattern data
+2. The pattern is generated by training ARToolKit on a physical marker
+3. The pattern file contains a 16x16 grid of values for pattern matching
+
+## Code Overview
+
+Key parts of the example code:
+
+```javascript
+// Create plugin instance with worker enabled
+const plugin = new ArtoolkitPlugin({ worker: true });
+
+// Initialize and enable
+await plugin.init(mockCore);
+await plugin.enable();
+
+// Load a pattern marker
+const result = await plugin.loadMarker('/examples/simple-marker/data/patt.hiro', 1);
+console.log(`Marker loaded with ID: ${result.markerId}`);
+```
+
+## Next Steps
+
+- Integrate with an actual ARToolKit WASM module for real marker detection
+- Add camera/video input for live marker tracking
+- Visualize detected markers with 3D graphics (Three.js, Babylon.js, etc.)
+- Load multiple markers with different patterns
+
+## Troubleshooting
+
+**Worker not loading?**
+- Make sure you're serving the files through HTTP/HTTPS, not opening them directly with `file://`
+- Check browser console for any CORS or module loading errors
+
+**Marker not loading?**
+- Check the browser console for detailed logs
+- Verify the pattern file path is accessible
+- Ensure the worker is ready before calling `loadMarker()`
+
+## Browser Support
+
+This example requires:
+- ES modules support
+- Web Workers support
+- Modern browser (Chrome 80+, Firefox 75+, Safari 13.1+, Edge 80+)