A browser-based gesture drawing experience where you draw shapes in the air using hand gestures via your webcam. Completed drawings inflate into soft, 3D balloon-like objects that float in a shared scene.
Disclaimer: This project was built in about 1 hours, so expect bugs! If you encounter any issues or have suggestions, feel free to open an issue or leave feedback. I'd love to hear what you think! Also, feel free to fork this project and make your own version of it.
- Draggable Camera Preview - Move the camera preview anywhere on screen so it doesn't block your creations. Double-click to reset position.
- Improved Line Smoothing - Added jitter filter to eliminate hand vibration for smoother drawing
- Better Hand Detection - Switched to higher accuracy model for more reliable tracking
- Clear All Button - Replaced fist gesture with a dedicated button for easier clearing
- Expandable Preview - Click the expand button on camera preview to enlarge it
- Gesture-Based Drawing - Point your index finger to draw in the air
- 3D Balloon Inflation - Completed shapes transform into puffy, floating 3D objects
- Real-Time Hand Tracking - Powered by MediaPipe for responsive hand detection
- Color Palette - Choose from 10 pastel colors for your creations
- Interactive Objects - Poke, grab, and rotate your balloon creations
- Draggable Camera Preview - See your hand tracking skeleton and move it anywhere
- Mouse/Touch Controls - Orbit and zoom the 3D scene
- Draw - Extend your index finger (keep other fingers curled) to draw
- Complete Shape - Hold an open palm for 0.5 seconds to close and inflate your drawing
- Interact - Pinch to grab and move objects, poke with your finger to squish them
- Clear - Click the "Clear All" button to remove all objects
| Gesture | Action |
|---|---|
| Point (index finger) | Draw in the air |
| Open Palm (hold) | Close shape and inflate to 3D |
| Pinch | Grab and move objects |
| Swipe | Remove individual object |
| Control | Action |
|---|---|
| Clear All button | Remove all objects |
| Color swatches | Change drawing color |
| Camera preview | Drag to move, double-click to reset |
| Expand button (on preview) | Toggle larger preview |
# Clone the repository
git clone https://github.com/Ai-Haris/AirCanvas.git
cd AirCanvas
# Install dependencies
npm install
# Start development server
npm run devThen open your browser to the local URL shown in the terminal (usually http://localhost:5173).
- Modern browser with WebGL support (Chrome, Firefox, Edge, Safari)
- Webcam access
- Good lighting for hand tracking
- HTTPS connection (required for camera access when hosted online)
- TypeScript - Type-safe JavaScript
- Vite - Fast build tool and dev server
- Three.js - 3D rendering and scene management
- MediaPipe Hands - Real-time hand tracking
- GSAP - Smooth animations
- Click + Drag on empty space to orbit the camera
- Click + Drag on an object to rotate it
- Scroll wheel to zoom in/out
- Touch gestures supported on mobile
src/
├── main.ts # Application entry point
├── handTracking.ts # MediaPipe hand detection
├── gestureDetector.ts # Gesture recognition logic
├── drawingCanvas.ts # 2D stroke rendering
├── scene3D.ts # Three.js scene setup
├── objectManager.ts # 3D balloon creation and physics
├── balloonInflator.ts # 3D mesh generation from strokes
├── handVisualizer.ts # Hand skeleton overlay
├── multiplayer.ts # PeerJS multiplayer support
├── constants.ts # Configuration values
└── types.ts # TypeScript interfaces
- Use good lighting so your hand is clearly visible
- Keep your hand about 1-2 feet from the camera
- Point with just your index finger extended for drawing
- Draw slowly and steadily for smoother lines
Found a bug? Have an idea? I'd love to hear from you!
- Open an issue on GitHub
- Fork and submit a pull request
- Share your own version built on this project
Designed by Haris Ai
This project is built with these amazing open source libraries:
| Library | Description | License |
|---|---|---|
| Three.js | 3D graphics library for WebGL rendering | MIT |
| MediaPipe Hands | Google's real-time hand tracking solution | Apache 2.0 |
| GSAP | Professional-grade animation library | GreenSock License |
| PeerJS | WebRTC peer-to-peer connections made easy | MIT |
| Vite | Next-generation frontend build tool | MIT |
| TypeScript | Typed superset of JavaScript | Apache 2.0 |
Thank you to all the maintainers and contributors of these projects!
MIT