AirPiano

AirPiano is a gesture-controlled virtual piano built using Python, OpenCV, MediaPipe, and FluidSynth.
It allows users to play a piano in mid-air using hand and finger movements, without touching any physical keys.

This project was developed as an 11th grade Computer Science Exhibition submission and focuses on real-time computer vision, human–computer interaction, and audio synthesis.

---

Features

• Real-time virtual piano rendered on screen
• Hand tracking using MediaPipe
• Finger-based note triggering
• Multiple instruments (Piano, Organ, Synth, Flute, Strings, Violin)
• Multiple finger play modes
• Smooth note release damping (natural sound decay)
• On-screen Help / About me (press H)
• Polished UI with visual feedback
• High-quality sound using FluidSynth + SoundFont

---

Working

1. Webcam feed is captured using OpenCV
2. MediaPipe Hands detects hand landmarks in real time
3. Finger extension is detected using screen-space (Y-axis) logic
4. When a finger overlaps a piano key:
   • A MIDI note is triggered via FluidSynth
5. Notes are released with a small delay to simulate natural damping

---

Requirements

• Python 3.9+
• Webcam
• Windows (tested on Windows 11)

---

Installation

Install Python dependencies

pip install -r requirements.txt

SoundFont (Required)

This project uses the FluidR3_GM.sf2 SoundFont for audio synthesis.

Due to file size limitations, the SoundFont is not included in this repository.

Download

Download the SoundFont from the following link:

https://drive.google.com/file/d/0B4Psp5O5pxlLc0pXdUh5RVNfMjQ/view?pli=1&resourcekey=0-FhpU9VBCyBdd9YiGgAuXVw

Setup

After downloading:

1. Rename the file to FluidR3_GM.sf2 (if required)
2. Place it in the same folder as main.py

Install FluidSynth (Windows)

Run PowerShell as Administrator, then:

choco install fluidsynth

Requires Chocolatey If Chocolatey is not installed, install it from: https://chocolatey.org

---

▶️ Running the Project

python -u main.py

---

Controls

Key	Action
A / D	Change instrument
W / S	Change finger mode
H	Toggle Help / About screen
Q	Quit application

---

Files

• main.py – Main application logic
• requirements.txt – Python dependencies
• FluidR3_GM.sf2 – General MIDI SoundFont (required)

---

Notes

• Ensure FluidR3_GM.sf2 is present in the project directory
• Use good lighting for better hand tracking accuracy
• A plain background improves detection stability

---

Sources & Acknowledgements

• MediaPipe Hands – real-time hand tracking
• OpenCV – video processing and rendering
• FluidSynth – MIDI audio synthesis
• SoundFont: FluidR3 GM

---

Credits

Built with ♥️ by @codyandersan