Add ECS architecture and plugin system with source capture plugins

[Copilot]

ECS Architecture and Plugin System Implementation — COMPLETE ✅

Summary

This PR implements a modular, data‑oriented ECS (Entity‑Component‑System) core with a plugin system for AR.js Core and keeps backward compatibility. It now also integrates a working ArtoolkitPlugin example with UI controls, a frame pump to stream ImageBitmaps to the worker, a visible video viewport, and new docs/troubleshooting.

Highlights

• 🧩 ECS core + plugin system with capture/profile plugins
• 🎛️ Example app with Start Webcam, Stop, and Load Marker (HIRO) buttons
• 🎥 Live video viewport (shows the webcam feed on page)
• 🔁 Frame pump streams ImageBitmaps via engine:update for reliable detection
• 🧠 Marker events: ar:markerFound / ar:markerUpdated / ar:markerLost
• 🧹 Lint‑safe browser globals via globalThis guards
• 📄 Docs: example README, “Frame Pump + Video Viewport” section, and Troubleshooting

---

What Was Delivered

Core ECS Infrastructure ✅

• src/core/ecs.js — Minimal ECS with entities, components, resources, and queries
• src/core/event-bus.js — Lightweight pub/sub
• src/core/plugin-manager.js — Plugin registration + lifecycle
• src/core/engine.js — Orchestrates ECS, events, plugins, loop
• src/core/components.js — Standard component/resource/event keys

Systems ✅

• src/systems/capture-system.js — Unified capture interface and lifecycle
• src/systems/frame-pump-system.js — Emits engine:update with ImageBitmap frames
  • Prefers HTMLVideoElement.requestVideoFrameCallback
  • Falls back to requestAnimationFrame
  • Uses globalThis.createImageBitmap(video) fast path; canvas fallback
  • Cancels via video.cancelVideoFrameCallback(id) or globalThis.cancelAnimationFrame

Plugins ✅

• plugins/source/webcam.js — getUserMedia capture (+ torch)
• plugins/source/video.js — Video file/URL playback
• plugins/source/image.js — Static image source
• plugins/profile/default-policy.js — Device profile/policy

Example Applications ✅

• examples/minimal/ — Base demo (status, FPS, error handling)
• examples/vite-artoolkit/ — AR.js Core ECS + ArtoolkitPlugin example
  • Loads ArtoolkitPlugin ESM (vendor or CDN)
  • UI: Start Webcam, Stop, Load Marker (HIRO)
  • Attaches <video id="arjs-video"> to a visible viewport
  • Starts FramePumpSystem to stream ImageBitmaps
  • Listens for ar:workerReady, ar:getMarker, and marker events

Documentation ✅

• examples/vite-artoolkit/README.md — How to run, buttons, viewport, troubleshooting
• Main README.md
  • “Frame Pump + Video Viewport” section
  • “Troubleshooting” section (common 404s, worker assets, camera params, pattern paths)
• docs/ECS_ARCHITECTURE.md — Architecture guide
• docs/IMPLEMENTATION_SUMMARY.md — Implementation summary
• plugins/README.md — Plugin development guide

Build & Configuration ✅

• ESM module support
• Dev tooling and lint formatting
• Proper .gitignore
• Webpack output filename arjs-core.js (library name ARJSCore)
• three.js as external

---

Why This Matters

• 🔌 Modular architecture: plugins and systems are easy to extend
• 🧵 Off‑main‑thread detection with a clear frame pipeline
• 🧭 Example shows end‑to‑end integration: capture → viewport → frame pump → detection
• 🧰 Clear troubleshooting for asset paths and worker setups

---

How to Run the Artoolkit Example

1. Serve from repository root (so /examples and vendor assets resolve):
   • python3 -m http.server 8080
   • or npx http-server -p 8080
2. Open: http://localhost:8080/examples/vite-artoolkit/index.html
3. Click “Start Webcam”, then “Load Marker”, and present the HIRO pattern
   • Expect ar:getMarker logs and markerFound/Updated/Lost events

Notes

• Vendor ESM path must include its assets/ folder (worker + ARToolKit chunks), or import from CDN:

  const mod = await import('https://cdn.jsdelivr.net/gh/AR-js-org/arjs-plugin-artoolkit@main/dist/arjs-plugin-artoolkit.esm.js');

---

Troubleshooting (quick)

• Worker/ARToolKit chunks 404 → ensure vendor ESM and its assets/ folder are colocated or use the CDN import
• camera_para.dat / patt.hiro must return 200
• No detections → ensure FramePumpSystem is running (emits engine:update with ImageBitmap)
• Video not visible → attach <video id="arjs-video"> to a visible container and override offscreen styles
• Lint → guard browser APIs via globalThis and cancel rVFC via video.cancelVideoFrameCallback(id)

---

Backward Compatibility

• Legacy API preserved and exported
• New ECS and plugins are additive, non‑breaking

Original prompt

Goal
Introduce a modular, data-oriented core to AR.js-core using an ECS (Entity-Component-System) architecture and a plugin system. Provide initial source capture plugins (webcam, video, image), a default profile policy plugin, a capture system, and a minimal example app. Keep current classes (arjs-source.js, arjs-profile.js) intact but add legacy adapters that map the old API to the new internals to preserve backward compatibility. Add basic build wiring (ESM + dev server) and documentation.

Scope of this PR

1. Core (new files)

• src/core/ecs.js: Minimal ECS with entities, components, queries.
• src/core/event-bus.js: Lightweight pub/sub event bus.
• src/core/plugin-manager.js: Register/enable/disable plugins.
• src/core/engine.js: Wires ECS, events, plugin manager, game loop, and systems.
• src/core/components.js: Declare shared component/resource keys (ProcessingConfig, CaptureState, FrameSourceRef, DeviceProfile).

2. Systems (new)

• src/systems/capture-system.js: Initializes the chosen capture plugin, writes FrameSourceRef + CaptureState resources, emits lifecycle events.

3. Plugins (new)

• plugins/source/webcam.js: getUserMedia-based capture, emits source lifecycle events, provides HTMLVideoElement/MediaStream.
• plugins/source/video.js: Local/remote
• plugins/source/image.js: Static source.
• plugins/profile/default.js: Computes device class and writes ProcessingConfig and DeviceProfile resources; emits profile:updated.

4. Legacy adapters (new)

• adapters/legacy/arjs-source-adapter.js: Exposes the old Source class API; internally spins up an engine, registers capture plugins, and mirrors the old DOM behavior (appends element with styles and id="arjs-video").
• adapters/legacy/arjs-profile-adapter.js: Exposes the old Profile class API; maps performance presets and defaults while deferring backend-specific URLs to tracker plugins (to be implemented in future PRs).

5. Minimal example (new)

• examples/minimal/index.html: Simple HTML page with a container and a notice for user gesture if needed.
• examples/minimal/main.js: Creates engine, registers plugins, starts capture (webcam by default), and appends the video element to the page for visual confirmation. Demonstrates using the legacy Source adapter as well as the new API.

6. Build/Dev wiring

• package.json updates/additions:
  • Module type: ESM.
  • Scripts: "dev" (Vite), "build" (Vite build to dist), "serve" (preview), "lint".
  • Add devDependencies: vite (and optionally vite-plugin-basic-ssl for HTTPS dev on webcam).
• vite.config.js: Simple ESM build and a dev server with HTTPS option to allow getUserMedia in local dev.
• .editorconfig and .eslintrc: basic code style (optional in this PR).

7. Docs

• README.md: High-level architecture overview (ECS + plugins), how to run the minimal example, and how to continue using legacy Source/Profile via adapters.
• docs/plugins.md: Plugin contract (id, init, lifecycle hooks) and examples for capture/profile plugins; document events and ECS components.

Acceptance criteria

• Build runs: npm i && npm run dev starts a dev server where examples/minimal works (webcam permission flow visible) and shows a playing video in the page (or a message if blocked until click).
• npm run build generates ESM bundles under dist/ without bundling peer dependencies (three.js/ARToolkit are not introduced in this PR).
• Legacy adapters work: importing adapters/legacy/arjs-source-adapter.js allows existing initialization patterns to display the source element.
• No breaking changes to existing files (e.g., src/arjs-source.js and src/arjs-profile.js remain as-is). New code is additive.
• Lint passes for newly added files (basic rules if configured).

Implementation notes / skeletons to include

• Implement the skeleton files as previously discussed and wire them in the example. Ensure plugin IDs match between registration and usage.
• Prefer small, readable functions; errors should emit events and propagate to onError callbacks.
• Keep DOM side effects in example/adapter layers; core plugins should primarily produce frame resources and events.

Non-goals for this PR

• No tracker/renderer plugins (ARToolkit, three.js, A-Frame) yet — to be added in follow-up PRs.
• No removal/refactor of existing arjs-source.js or arjs-profile.js; only additive changes and adapters.

Branch

• Create branch name: feat/modular-core-skeleton and target base branch: main.

This pull request was created as a result of the following prompt from Copilot chat.

Goal
Introduce a modular, data-oriented core to AR.js-core using an ECS (Entity-Component-System) architecture and a plugin system. Provide initial source capture plugins (webcam, video, image), a default profile policy plugin, a capture system, and a minimal example app. Keep current classes (arjs-source.js, arjs-profile.js) intact but add legacy adapters that map the old API to the new internals to preserve backward compatibility. Add basic build wiring (ESM + dev server) and documentation.

Scope of this PR

1. Core (new files)

• src/core/ecs.js: Minimal ECS with entities, components, queries.
• src/core/event-bus.js: Lightweight pub/sub event bus.
• src/core/plugin-manager.js: Register/enable/disable plugins.
• src/core/engine.js: Wires ECS, events, plugin manager, game loop, and systems.
• src/core/components.js: Declare shared component/resource keys (ProcessingConfig, CaptureState, FrameSourceRef, DeviceProfile).

2. Systems (new)

• src/systems/capture-system.js: Initializes the chosen capture plugin, writes FrameSourceRef + CaptureState resources, emits lifecycle events.

3. Plugins (new)

• plugins/source/webcam.js: getUserMedia-based capture, emits source lifecycle events, provides HTMLVideoElement/MediaStream.
• plugins/source/video.js: Local/remote
• plugins/source/image.js: Static source.
• plugins/profile/default.js: Computes device class and writes ProcessingConfig and DeviceProfile resources; emits profile:updated.

4. Legacy adapters (new)

• adapters/legacy/arjs-source-adapter.js: Exposes the old Source class API; internally spins up an engine, registers capture plugins, and mirrors the old DOM behavior (appends element with styles and id="arjs-video").
• adapters/legacy/arjs-profile-adapter.js: Exposes the old Profile class API; maps performance presets and defaults while deferring backend-specific URLs to tracker plugins (to be implemented in future PRs).

5. Minimal example (new)

• examples/minimal/index.html: Simple HTML page with a container and a notice for user gesture if needed.
• examples/minimal/main.js: Creates engine, registers plugins, starts capture (webcam by default), and appends the video element to the page for visual confirmation. Demonstrates using the legacy Source adapter as well as the new API.

6. Build/Dev wiring

• package.json updates/additions:
  • Module type: ESM.
  • Scripts: "dev" (Vite), "build" (Vite build to dist), "serve" (preview), "lint".
  • Add devDependencies: vite (and optionally vite-plugin-basic-ssl for HTTPS dev on webcam).
• vite.config.js: Simple ESM build and a dev server with HTTPS option to allow getUserMedia in local dev.
• .editorconfig and .eslintrc: basic code style (optional in this PR).

7. Docs

• README.md: High-level architecture overview (ECS + plugins), how to run the minimal example, and how to continue using legacy Source/Profile via adapters.
• docs/plugins.md: Plugin contract (id, init, lifecycle hooks) and examples for capture/profile plugins; document events and ECS components.

Acceptance criteria

• Build runs: npm i && npm run dev starts a dev server where examples/minimal works (webcam permission flow visible) and shows a playing video in the page (or a message if blocked until click).
• npm run build generates ESM bundles under dist/ without bundling peer dependencies (three.js/ARToolkit are not introduced in this PR).
• Legacy adapters work: importing adapters/legacy/arjs-source-adapter.js allows existing initialization patterns to display the source element.
• No breaking changes to existing files (e.g., src/arjs-source.js and src/arjs-profile.js remain as-is). New code is additive.
• Lint passes for newly added files (basic rules if configured).

Implementation notes / skeletons to include

• Implement the skeleton files as previously discussed and wire them in the example. Ensure plugin IDs match between registration and usage.
• Prefer small, readable functions; errors should emit events and propagate to onError callbacks.
• Keep DOM side effects in example/adapter layers; core plugins should primarily produce frame resources and events.

Non-goals for this PR

• No tracker/renderer plugins (ARToolkit, three.js, A-Frame) yet — to be added in follow-up PRs.
• No removal/refactor of existing arjs-source.js or arjs-profile.js; only additive changes and adapters.

Branch

• Create branch name: feat/modular-core-skeleton and target base branch: main.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.