rewrite solver and migrate to nextjs

Author: AmyangXYZ

.gitignore

@@ -1,29 +1,41 @@
-# Logs
-logs
-*.log
+# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
+
+# dependencies
+/node_modules
+/.pnp
+.pnp.*
+.yarn/*
+!.yarn/patches
+!.yarn/plugins
+!.yarn/releases
+!.yarn/versions
+
+# testing
+/coverage
+
+# next.js
+/.next/
+/out/
+
+# production
+/build
+
+# misc
+.DS_Store
+*.pem
+
+# debug
 npm-debug.log*
 yarn-debug.log*
 yarn-error.log*
-pnpm-debug.log*
-lerna-debug.log*
-
-node_modules
-dist
-dist-ssr
-dev-dist
-*.local
-
-# Editor directories and files
-.vscode/*
-!.vscode/extensions.json
-.idea
-.DS_Store
-*.suo
-*.ntvs*
-*.njsproj
-*.sln
-*.sw?
+.pnpm-debug.log*
+
+# env files (can opt-in for committing if needed)
+.env*
 
+# vercel
 .vercel
 
-pose_solver/target
+# typescript
+*.tsbuildinfo
+next-env.d.ts

LICENSE

@@ -1,61 +1,84 @@
-# MiKaPo: AI Pose Picker for MikuMikuDance
+# MiKaPo: Real-time MMD Motion Capture
 
-> **🎉 NEW PROJECT ALERT!** Check out [**PoPo**](https://popo.love) - Transform text into MMD poses with AI! No more manual bone adjustments - just type "shy smile while waving" and watch the magic happen ✨
+A web-based tool that enables real-time motion capture for MikuMikuDance (MMD) models.
 
-<img width="300px" alt="demo_pose" src="./logo.jpg" />
+## Overview
 
-[MiKaPo](https://mikapo.amyang.dev) is a **Web-based tool** that poses MMD models from video input in real-time. Welcome feature requests and PRs!
+[MiKaPo](https://mikapo.amyang.dev) transforms video input into real-time MMD model poses by detecting 3D landmarks and converting them to bone rotations. The core technical challenge lies in accurately mapping world-space 3D landmarks from MediaPipe to MMD bone quaternion rotations, accounting for MMD's specific bone coordinate system and directional conventions.
 
-<img width="400px" alt="demo_pose" src="./demo1.gif" />
-<img width="400px" alt="demo_face" src="./demo2.gif" />
-<img width="400px" alt="demo_img" src="./demo3.png" />
+**MiKaPo 2.0** introduces a completely rewritten solver with hierarchical bone transformations, migrating from Vite to Next.js for improved performance and maintainability.
 
-## Tech Stack
+![](./screenshots/1.png)
+![](./screenshots/2.png)
 
-- 3D key points detection: [Mediapipe](https://ai.google.dev/edge/mediapipe/solutions/vision/pose_landmarker/web_js)
-- 3D scene: [Babylon.js](https://www.babylonjs.com/)
-- MMD model viewer: [babylon-mmd](https://github.com/noname0310/babylon-mmd)
-- Web framework: [Vite+React](https://vitejs.dev/)
-- Models are from [aplaybox](https://aplaybox.com/en/mmd-models/).
+## Related Project
 
-## Features
+Check out [**PoPo**](https://popo.love) - AI-powered text-to-MMD pose generation. Transform natural language descriptions into MMD poses instantly.
 
-- [x] Pose detection
-- [x] Face detection
-- [x] Hand detection (experimental)
-- [x] Rust-WASM based pose-to-quaternion solver
-- [x] 360-degree background selection
-- [x] Video, image upload
-- [x] Webcam input
-- [x] Model selection
-- [x] Ollama support ([electron version](https://github.com/AmyangXYZ/MiKaPo-Electron))
-- [x] VMD import/export (to export a valid VMD file, you must record at least one motion)
-- [x] MMD editor: bone, material, mesh edit
+## Key Features
 
-## Hint
+- **Real-time pose detection** using MediaPipe Pose
+- **Face and hand tracking** for comprehensive motion capture
+- **Multiple input sources**: webcam, video files, and image uploads
+- **Live MMD model rendering** with synchronized bone animations
 
-- Let your browser use dedicated GPU for better performance.
+_Legacy features from v1.0 (VMD export, bone manipulation, 360° scene environment) will be added in future updates._
 
-## Project Setup
+## Technical Stack
 
-```sh
-npm install
-```
+- **3D Pose Detection**: [MediaPipe Pose Landmarker](https://ai.google.dev/edge/mediapipe/solutions/vision/pose_landmarker/web_js)
+- **3D Graphics Engine**: [Babylon.js](https://www.babylonjs.com/)
+- **MMD Integration**: [babylon-mmd](https://github.com/noname0310/babylon-mmd)
+- **Web Framework**: [Next.js](https://nextjs.org/)
 
-### Compile and Hot-Reload for Development
+## Core Challenge
 
-```sh
-npm run dev
-```
+The primary technical challenge involves solving the complex transformation from world-space 3D landmarks to MMD bone quaternion rotations. This requires:
 
-### Type-Check, Compile and Minify for Production
+- Converting MediaPipe's coordinate system to MMD's bone space
+- Handling MMD's unique bone direction conventions
+- Computing accurate quaternion rotations for smooth animations
+- Maintaining temporal consistency across frames
 
-```sh
-npm run build
-```
+## Technical Solution
+
+The solver implements a hierarchical transformation approach that maps MediaPipe's world-space landmarks to MMD bone rotations:
+
+```typescript
+// Key Algorithm Pseudocode
+function solveBoneRotation(landmarkName: string, parentChain: string[]): Quaternion {
+  // 1. Get world-space landmarks from MediaPipe
+  const worldLandmark = getMediaPipeLandmark(landmarkName)
+  const worldTarget = getMediaPipeLandmark(targetLandmarkName)
+
+  // 2. Build full parent bone hierarchy chain (not just immediate parent)
+  const fullParentQuat = parentChain.reduce(
+    (acc, parent) => acc.multiply(boneStates[parent].rotation),
+    Quaternion.Identity()
+  )
 
-### Lint with [ESLint](https://eslint.org/)
+  // 3. Transform world landmarks to parent's local space
+  const parentMatrix = Matrix.FromQuaternion(fullParentQuat).invert()
+  const localLandmark = Vector3.TransformCoordinates(worldLandmark, parentMatrix)
+  const localTarget = Vector3.TransformCoordinates(worldTarget, parentMatrix)
 
-```sh
-npm run lint
+  // 4. Calculate bone direction in local space
+  const boneDirection = localTarget.subtract(localLandmark).normalize()
+
+  // 5. Set MMD bone's default A-pose reference direction
+  const mmdReferenceDirection = getMMDDefaultDirection(boneName)
+
+  // 6. Compute quaternion rotation from reference to current direction
+  return Quaternion.FromUnitVectors(referenceDirection, boneDirection)
+}
+
+// Example: Left wrist transformation chain
+// Parent hierarchy: upper_body → left_arm → left_elbow → left_wrist
+// Each bone's rotation is computed in its parent's local space
 ```
+
+This approach ensures accurate bone rotations by:
+
+- **Hierarchical Transformation**: Each bone is solved in its full parent chain's local space
+- **MMD A-Pose Alignment**: Reference directions match MMD's default bone orientations
+- **Coordinate System Conversion**: Properly handles MediaPipe's coordinate system to MMD's bone space

README.md

@@ -0,0 +1,21 @@
+{
+  "$schema": "https://ui.shadcn.com/schema.json",
+  "style": "new-york",
+  "rsc": true,
+  "tsx": true,
+  "tailwind": {
+    "config": "",
+    "css": "src/app/globals.css",
+    "baseColor": "zinc",
+    "cssVariables": true,
+    "prefix": ""
+  },
+  "aliases": {
+    "components": "@/components",
+    "utils": "@/lib/utils",
+    "ui": "@/components/ui",
+    "lib": "@/lib",
+    "hooks": "@/hooks"
+  },
+  "iconLibrary": "lucide"
+}

components.json

@@ -0,0 +1,16 @@
+import { dirname } from "path";
+import { fileURLToPath } from "url";
+import { FlatCompat } from "@eslint/eslintrc";
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = dirname(__filename);
+
+const compat = new FlatCompat({
+  baseDirectory: __dirname,
+});
+
+const eslintConfig = [
+  ...compat.extends("next/core-web-vitals", "next/typescript"),
+];
+
+export default eslintConfig;