Integrate the WebARKit OCVT tracker line: dev → master

.github/workflows/test.yml

@@ -0,0 +1,26 @@
+name: WebARKitLib test
+
+on: push
+jobs:
+    build:
+        runs-on: ubuntu-latest
+        steps:
+        - uses: actions/checkout@v4
+        - name: Setup Node.js
+          uses: actions/setup-node@v4
+          with:
+            node-version: '20.x'
+        - name: Update Ubuntu and install libjpeg-dev
+          run: |
+              sudo apt-get update && sudo apt install libjpeg-dev
+        - name: Build and test WebARKitLib
+          run: |
+              cd tests && mkdir build && cd build && cmake -DEMSCRIPTEN_COMP=0 .. && make && ./webarkit_test
+        - name: Build WebARKitLib with Emscripten (Docker)
+          run: |
+             cd ..
+             ls
+             docker run -dit --name emscripten-webarkit-testing -v $(pwd):/src emscripten/emsdk:3.1.38 bash
+             docker exec emscripten-webarkit-testing emcmake cmake -B WebARKitLib/WebARKit/build -S WebARKitLib/WebARKit -DEMSCRIPTEN_COMP=1 ..
+             docker exec emscripten-webarkit-testing emmake make -C WebARKitLib/WebARKit/build
+

.gitignore

@@ -1 +1,3 @@
-include/AR/config.h
+include/AR/config.h
+tests/build
+build

CONTRIBUTING.md

@@ -0,0 +1,73 @@
+# Contributing to WebARKitLib
+
+Thanks for helping improve WebARKit! This is the C/C++ source of the tracker; most
+changes are consumed by the [webarkit-testing](https://github.com/webarkit/webarkit-testing)
+superproject as a submodule.
+
+## Workflow
+
+- **Branch from `dev`** and open your PR **against `dev`** (not `main`).
+- Reference the related issue in the PR description.
+- **Sign your commits** (`git commit -S …`).
+
+## Commit messages — Conventional Commits
+
+All commit messages **must** follow
+[Conventional Commits](https://www.conventionalcommits.org/):
+
+```
+<type>(<optional scope>): <short description>
+```
+
+Common types: `feat`, `fix`, `docs`, `refactor`, `test`, `chore`, `perf`. Examples:
+
+```
+feat(tracker): detect features on a downsampled frame
+fix(#55): populate pose3d so getPoseMatrixCV() isn't zero
+docs: document the pose pipeline
+refactor(#50): remove dead computePose/invertPose
+```
+
+## What you may change
+
+- Active WebAR work lives under **`WebARKit/`** (the OCVT planar tracker).
+- **Do not** modify the vendored **old ARToolKit5** sources under `lib/SRC/**` or
+  `include/AR/**` — those are upstream utilities used by
+  [jsartoolkitNFT](https://github.com/webarkit/jsartoolkitNFT) /
+  [ARnft](https://github.com/webarkit/ARnft) and are kept untouched.
+
+## Cross-repo changes (the PR pair)
+
+> **Applies to `WebARKit/` (this folder) ↔ the `webarkit-testing` repo.** This flow is
+> for changes to the **`WebARKit/`** OCVT tracker, which the **webarkit-testing**
+> superproject compiles to WASM. Changes to the vendored `lib/SRC/**` / `include/AR/**`
+> (old ARToolKit5) do **not** use this flow — they reach the web through
+> [jsartoolkitNFT](https://github.com/webarkit/jsartoolkitNFT), a different consumer.
+
+A change to `WebARKit/` that affects the WASM build is consumed by webarkit-testing via
+the submodule. Coordinate both repos:
+
+1. Branch from `dev` in **both** WebARKitLib and webarkit-testing.
+2. Make the C++ change here; in webarkit-testing bump the submodule pointer and
+   rebuild (`npm run build-docker` → `build-es6`).
+3. Open a **PR pair** (WebARKitLib → `dev`, webarkit-testing → `dev`); cross-link them.
+4. After the WebARKitLib PR merges, re-point the submodule to the merged `dev` tip.
+
+## Building & tests
+
+- The library is normally built **from webarkit-testing** (WASM) — see its README.
+  `WebARKit/CMakeLists.txt` can also build it as a static lib for **WASM** or **native
+  Linux** (selected via `EMSCRIPTEN_COMP`), pulling OpenCV from
+  [webarkit/opencv-em](https://github.com/webarkit/opencv-em).
+- **Unit tests** (GoogleTest) live in `tests/` and run in CI on every PR:
+
+  ```bash
+  cmake -S tests -B tests/build && cmake --build tests/build && ctest --test-dir tests/build
+  ```
+
+  If you change a config constant or the version, update the matching assertions in
+  `tests/webarkit_test.cc`.
+
+## License
+
+By contributing, you agree that your contributions are licensed under **LGPL-3.0**.

README.md

@@ -1,4 +1,98 @@
+![github releases](https://flat.badgen.net/github/release/webarkit/WebARKitLib)
+![github stars](https://flat.badgen.net/github/stars/webarkit/WebARKitLib)
+![github forks](https://flat.badgen.net/github/forks/webarkit/WebARKitLib)[![Test](https://github.com/webarkit/WebARKitLib/actions/workflows/test.yml/badge.svg)](https://github.com/webarkit/WebARKitLib/actions/workflows/test.yml)
+
 # WebARKitLib
 
-The C/C++ source code of WebARKit, from Artoolkit5 and extended.
-Updated to [Eigen](https://eigen.tuxfamily.org) 3.4.0 and  Emscripten emsdk 3.1.26.
+The C/C++ source of **WebARKit** — a browser-side Augmented Reality tracker. The
+active code (`WebARKit/`) is an emscripten-friendly port of the **OCVT** planar
+image-tracking design from [ArtoolkitX](https://github.com/artoolkitx/artoolkitx),
+built on OpenCV. It is normally compiled to WebAssembly and consumed from the
+[webarkit-testing](https://github.com/webarkit/webarkit-testing) superproject.
+
+> **Two lineages live here, keep them straight:**
+> - **`WebARKit/`** — the WebAR OCVT planar tracker (the design reference is *ArtoolkitX*). This is the code under active development.
+> - **`lib/SRC/**`, `include/AR/**`** — vendored **old ARToolKit5** utility sources/headers, kept only because [jsartoolkitNFT](https://github.com/webarkit/jsartoolkitNFT) needs them (and [ARnft](https://github.com/webarkit/ARnft) in turn consumes jsartoolkitNFT as an npm module). Treated as untouched upstream. (Updated to Eigen 3.4.0 / emsdk 3.1.26.)
+
+## Code layout (`WebARKit/`)
+
+| Component | Role |
+|---|---|
+| `WebARKitManager` | top-level façade — owns the tracker, drives init/process, exposes the output (homography, pose, GL-view, projection) |
+| `WebARKitTrackers/WebARKitOpticalTracking/WebARKitTracker` | the OCVT tracker core: feature detect/match → homography → optical-flow + template-match tracking |
+| `…/TrackingPointSelector` + `…/TrackedPoint` | bin-based selection of template tracking points (ArtoolkitX-derived) |
+| `…/WebARKitHomographyInfo` + `WebARKitUtils.h` | RANSAC homography estimation/validation (`getHomographyInliers`) |
+| `…/WebARKitConfig` | tuning constants (feature counts, match ratios, pyramid/template sizes, version) |
+| `…/TrackerVisualization` | optional debug-overlay scaffolding (inert until wired) |
+| `WebARKitCamera` | camera intrinsics from a diagonal-FOV estimate |
+| `WebARKitGL` | CV→GL matrix conversion (`arglCameraViewRHf`) + GL projection |
+| `WebARKitPattern` | the reference marker (`WebARKitPattern`) and per-frame pose state (`WebARKitPatternTrackingInfo`) |
+| `WebARKitLog` | logging |
+
+## Pose pipeline (one glance)
+
+```
+solvePnP (cameraPoseFromPoints)            // OpenCV-convention camera pose
+  -> getTrackablePose / updateTrackable    // CV->GL handedness fix, D*R*D  (see #42)
+  -> arglCameraViewRHf                      // right-handed GL modelview
+  -> matrixGL_RH                            // what the examples attach content to
+```
+
+Pose getters: **`getPoseMatrixCV()`** (raw 4×4 OpenCV pose) and **`getPoseMatrixGL()`**
+(the GL/right-handed pose); projection via `getCameraProjectionMatrix()`. The full
+derivation is in
+[`docs/design-projection-and-pose-artoolkitx-alignment.md`](https://github.com/webarkit/webarkit-testing/blob/dev/docs/design-projection-and-pose-artoolkitx-alignment.md)
+(webarkit-testing).
+
+## Building
+
+This repo isn't built as a single unit — each consumer compiles the **subset it
+needs**:
+
+- **`WebARKit/` (the OCVT tracker)** is built by the
+  [webarkit-testing](https://github.com/webarkit/webarkit-testing) superproject, which
+  compiles it to WebAssembly with emscripten (emsdk **3.1.26**) inside Docker and wires
+  it to JS through `emscripten/WebARKitJS.{cpp,h}` + `bindings.cpp` (which live in
+  webarkit-testing). webarkit-testing currently drives that compile with the Node script
+  `tools/makem.js` (`npm run build-docker` → `build-es6`); migrating to the CMake config
+  (`WebARKit/CMakeLists.txt`) is planned.
+- **The vendored `lib/SRC/**` + `include/AR/**` (old ARToolKit5)** is built by
+  [jsartoolkitNFT](https://github.com/webarkit/jsartoolkitNFT) as part of its build —
+  not by webarkit-testing. ([ARnft](https://github.com/webarkit/ARnft) consumes
+  jsartoolkitNFT as an npm module and doesn't build the emscripten code itself.)
+
+`WebARKit/CMakeLists.txt` can **already build WebARKitLib as a static library**
+today — for both **WASM** (emscripten) and **native Linux**, selected via the
+`EMSCRIPTEN_COMP` flag. It depends on a prebuilt **OpenCV** from
+[webarkit/opencv-em](https://github.com/webarkit/opencv-em) (the emscripten
+`opencv-js-…-emcc` build for WASM, or the native `opencv-…` build for Linux),
+fetched automatically via CMake `FetchContent`. The "planned" part above is
+webarkit-testing switching its WASM build over to this CMake config (away from
+`tools/makem.js`). The same config also drives the unit tests (see below).
+
+## Tests
+
+C++ unit tests (GoogleTest) live in [`tests/`](tests/) (`webarkit_test.cc`,
+`CMakeLists.txt`, `pinball.jpg`) and run in CI via
+[`.github/workflows/test.yml`](https://github.com/webarkit/WebARKitLib/actions/workflows/test.yml).
+Build them standalone with CMake:
+
+```bash
+cmake -S tests -B tests/build && cmake --build tests/build && ctest --test-dir tests/build
+```
+
+## Contributing
+
+Please read [`./CONTRIBUTING.md`](./CONTRIBUTING.md) for the full guidelines. In short:
+
+- Branch from `dev`; PR back to `dev`; sign your commits and reference the issue.
+- **Commit messages must follow [Conventional Commits](https://www.conventionalcommits.org/)**
+  (e.g. `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`).
+- Library changes are usually paired with a build/bump in webarkit-testing — see that
+  repo's "cross-repo PR pair" flow.
+- Don't modify the vendored `lib/SRC/**` / `include/AR/**` (old ARToolKit5) as part of
+  WebAR work.
+
+## License
+
+LGPL-3.0 (see `LICENSE.txt`).

WebARKit/CMakeLists.txt

@@ -0,0 +1,83 @@
+cmake_minimum_required(VERSION 3.16)
+
+project(WebARKitLib)
+
+set(CMAKE_CXX_STANDARD 14)
+
+if(VERSION GREATER 3.24)
+  cmake_policy(SET CMP0135 NEW)
+endif()
+
+include(FetchContent)
+
+if(${EMSCRIPTEN_COMP} EQUAL 1)
+  message("Fetching opencv for emscripten compilation from webarkit/opencv-em ...")
+  FetchContent_Declare(
+    build_opencv
+    URL https://github.com/webarkit/opencv-em/releases/download/0.1.6/opencv-js-4.10.0-emcc-3.1.38.zip
+  )
+else()
+  message("Fetching opencv from webarkit/opencv-em ...")
+  FetchContent_Declare(
+    build_opencv
+    URL https://github.com/webarkit/opencv-em/releases/download/0.1.6/opencv-4.10.0.zip
+  )
+endif()
+
+FetchContent_MakeAvailable(build_opencv)
+
+get_filename_component(PARENT_DIR ./ ABSOLUTE)
+
+set(WEBARKIT_HEADERS
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/include/WebARKitTrackers/WebARKitOpticalTracking/TrackedPoint.h
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/include/WebARKitTrackers/WebARKitOpticalTracking/TrackingPointSelector.h
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/include/WebARKitTrackers/WebARKitOpticalTracking/WebARKitConfig.h
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/include/WebARKitTrackers/WebARKitOpticalTracking/WebARKitEnums.h
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/include/WebARKitTrackers/WebARKitOpticalTracking/WebARKitHomographyInfo.h
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/include/WebARKitTrackers/WebARKitOpticalTracking/WebARKitTracker.h
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/include/WebARKitTrackers/WebARKitOpticalTracking/WebARKitUtils.h
+${PARENT_DIR}/include/WebARKitCamera.h
+${PARENT_DIR}/include/WebARKitLog.h
+${PARENT_DIR}/include/WebARKitGL.h
+${PARENT_DIR}/include/WebARKitManager.h
+${PARENT_DIR}/include/WebARKitPattern.h
+)
+
+set(SOURCE 
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/TrackedPoint.cpp
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/TrackingPointSelector.cpp
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/WebARKitConfig.cpp
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/WebARKitHomographyInfo.cpp
+${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/WebARKitTracker.cpp
+${PARENT_DIR}/WebARKitCamera.cpp
+${PARENT_DIR}/WebARKitLog.cpp
+${PARENT_DIR}/WebARKitGL.cpp
+${PARENT_DIR}/WebARKitManager.cpp
+${PARENT_DIR}/WebARKitPattern.cpp
+)
+
+add_library(
+  WebARKitLib STATIC
+  ${WEBARKIT_HEADERS}
+  ${SOURCE}
+)
+
+target_include_directories(WebARKitLib PRIVATE "${PARENT_DIR}/include")
+target_include_directories(WebARKitLib PRIVATE "${PARENT_DIR}/WebARKitTrackers/WebARKitOpticalTracking/include")
+
+target_include_directories(WebARKitLib PRIVATE
+        "${build_opencv_SOURCE_DIR}"
+        "${build_opencv_SOURCE_DIR}/libs/opencv/include"
+        "${build_opencv_SOURCE_DIR}/libs/opencv/modules/calib3d/include"
+        "${build_opencv_SOURCE_DIR}/libs/opencv/modules/core/include"
+        "${build_opencv_SOURCE_DIR}/libs/opencv/modules/features2d/include"
+        "${build_opencv_SOURCE_DIR}/libs/opencv/modules/flann/include"
+        "${build_opencv_SOURCE_DIR}/libs/opencv/modules/imgproc/include"
+        "${build_opencv_SOURCE_DIR}/libs/opencv/modules/video/include"
+        "${build_opencv_SOURCE_DIR}/libs/opencv_contrib/modules/xfeatures2d/include"
+        )
+
+target_link_libraries(WebARKitLib
+jpeg
+"${build_opencv_SOURCE_DIR}/3rdparty/lib/liblibopenjp2.a"
+)

WebARKit/WebARKitCamera.cpp

@@ -0,0 +1,52 @@
+#include <WebARKitCamera.h>
+#include <WebARKitLog.h>
+#include <WebARKitTrackers/WebARKitOpticalTracking/WebARKitConfig.h>
+
+namespace webarkit {
+WebARKitCamera::WebARKitCamera() : xsize(-1), ysize(-1), diagonal_fov_degrees(70.0) { cmat.fill(0.0); }
+
+WebARKitCamera::~WebARKitCamera() {}
+
+bool WebARKitCamera::setupCamera(int width, int height) {
+    if (width <= 0 || height <= 0) {
+        return false;
+    }
+    xsize = width;
+    ysize = height;
+
+    setFocalLength(xsize, ysize);
+
+    cmat.at(0) = focal_length;
+    cmat.at(2) = 0.5 * xsize;
+    cmat.at(4) = focal_length;
+    cmat.at(5) = 0.5 * ysize;
+    cmat.at(8) = 1.0;
+    kc.fill(0.0);
+    return true;
+};
+
+void WebARKitCamera::printSettings() {
+    WEBARKIT_LOGi("WebARKit: Camera Size %d , %d\n", xsize, ysize);
+    WEBARKIT_LOGi("WebARKit: camera matrix = [%.2f  %.2f %.2f]\n", cmat[0], cmat[1], cmat[2]);
+    WEBARKIT_LOGi("                          [%.2f  %.2f %.2f]\n", cmat[3], cmat[4], cmat[5]);
+    WEBARKIT_LOGi("                          [%.2f  %.2f %.2f]\n", cmat[6], cmat[7], cmat[8]);
+    WEBARKIT_LOGi("WebARKit: kc = [%.4f %.4f %.4f %.4f %.4f %.4f]\n", kc[0], kc[1], kc[2], kc[3], kc[4], kc[5]);
+};
+
+std::array<double, 9> WebARKitCamera::getCameraData() const {
+    return cmat;
+}
+
+std::array<double, 6> WebARKitCamera::getDistortionCoefficients() const {
+   return kc;
+}
+
+void WebARKitCamera::setFocalLength(int width, int height) {
+    double diagonal_image_size;
+    double diagonal_fov_radians;
+     // simple routine to calculate focal length from diagonal field of view, and convert to camera matrix.
+    diagonal_image_size = std::pow(std::pow(width, 2.0) + std::pow(height, 2.0), 0.5);
+    diagonal_fov_radians = diagonal_fov_degrees * m_pi / 180.0;
+    focal_length = 0.5 * diagonal_image_size / std::tan(0.5 * diagonal_fov_radians);
+}
+} // namespace webarkit