Skip to content

rzru/nightingale

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

132 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Nightingale

Karaoke from any song in your music library, powered by neural networks.


Nightingale scans your music folder, separates lead vocals from instrumentals using the UVR Karaoke model (or Demucs), transcribes lyrics with word-level timestamps via WhisperX, and plays it all back with synchronized highlighting, pitch scoring, profiles, and dynamic backgrounds.

Ships as a single binary. No manual installation of Python, ffmpeg, or ML models required โ€” everything is downloaded and bootstrapped automatically on first launch.

Features

๐ŸŽค Stem Separation โ€” isolates lead vocals from instrumentals using the UVR Karaoke model (default) or Demucs, with adjustable guide vocal volume. The karaoke model preserves backing vocals in the instrumental for a more natural sound

๐Ÿ“ Word-Level Lyrics โ€” automatic transcription with alignment, or fetched from LRCLIB when available

๐ŸŽฏ Pitch Scoring โ€” real-time microphone input with pitch detection, star ratings, and per-song scoreboards

๐Ÿ‘ค Profiles โ€” create and switch between player profiles; scores are tracked per profile

๐ŸŽฌ Video Files โ€” drop video files (.mp4, .mkv, etc.) into your music folder; vocals are separated from the audio track and the original video plays as a synchronized background

๐ŸŒŒ 7 Background Themes โ€” 5 GPU shader backgrounds (Plasma, Aurora, Waves, Nebula, Starfield), Pixabay video backgrounds with 5 flavors (Nature, Underwater, Space, City, Countryside), plus automatic source video playback for video files

๐ŸŽฎ Gamepad Support โ€” full navigation and control via gamepad (D-pad, sticks, face buttons)

๐Ÿ“บ Adaptive UI Scaling โ€” scales to any resolution including 4K TVs

๐Ÿ“ฆ Self-Contained โ€” ffmpeg, uv, Python, PyTorch, and ML packages are all downloaded to ~/.nightingale/vendor/ on first run. Video backgrounds are pre-downloaded during setup so the first session is ready to go

Quick start

Download the latest release for your platform from the Releases page and run it. On first launch, Nightingale will set up its Python environment and download ML models โ€” this takes a few minutes and shows a progress screen.

macOS

macOS quarantines files downloaded from the internet. Since Nightingale isn't signed with an Apple Developer ID, Gatekeeper will block it with a message like "app is damaged and can't be opened". To fix this, remove the quarantine attribute after extracting:

xattr -cr Nightingale.app

Supported formats

Audio: .mp3, .flac, .ogg, .wav, .m4a, .aac, .wma. Video: .mp4, .mkv, .avi, .webm, .mov, .m4v.

Controls

Navigation

Action Keyboard Gamepad
Move Arrow keys D-pad / Left stick
Confirm / Select Enter A (South)
Back / Cancel Escape B (East) / Start
Switch panel Tab โ€”
Search songs Type to filter โ€”

Playback

Action Keyboard Gamepad
Pause / Resume Space Start
Exit to menu Escape B (East)
Toggle guide vocals G โ€”
Guide volume up/down + / - โ€”
Cycle background theme T โ€”
Cycle video flavor F โ€”
Toggle microphone M โ€”
Next microphone N โ€”
Toggle fullscreen F11 โ€”
Skip Intro / Skip Outro On-screen buttons A (South)

How it works

Audio or video file
        โ”‚
        โ–ผ
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
  โ”‚  UVR Karaoke /   โ”‚  โ”€โ”€โ–ถ  vocals.ogg + instrumental.ogg
  โ”‚  Demucs          โ”‚       (extracts audio track from videos)
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
        โ”‚
        โ–ผ
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
  โ”‚  LRCLIB          โ”‚  โ”€โ”€โ–ถ  Fetches synced lyrics if available
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
        โ”‚
        โ–ผ
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
  โ”‚  WhisperX        โ”‚  โ”€โ”€โ–ถ  Transcription + word-level alignment
  โ”‚  (large-v3)      โ”‚
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
        โ”‚
        โ–ผ
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
  โ”‚  Bevy App        โ”‚  โ”€โ”€โ–ถ  Plays instrumental + synced lyrics
  โ”‚  (Rust)          โ”‚       with pitch scoring & backgrounds
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜       (video files use source video as background)

Analysis results are cached at ~/.nightingale/cache/ using blake3 file hashes. Re-analysis only happens if the source file changes or is manually triggered.

Hardware

The Python analyzer uses PyTorch and auto-detects the best backend:

Backend Device Notes
CUDA NVIDIA GPU Fastest
MPS Apple Silicon macOS; WhisperX alignment falls back to CPU
CPU Any Slowest but always works

The UVR Karaoke model uses ONNX Runtime and enables CUDA acceleration automatically on NVIDIA GPUs, or CoreML on Apple Silicon.

A song typically takes 2โ€“5 minutes on GPU, 10โ€“20 minutes on CPU.

Data storage

Everything lives under ~/.nightingale/:

~/.nightingale/
โ”œโ”€โ”€ cache/              # Stems, transcripts, lyrics per song
โ”œโ”€โ”€ config.json         # App settings
โ”œโ”€โ”€ profiles.json       # Player profiles and scores
โ”œโ”€โ”€ videos/             # Cached Pixabay video backgrounds
โ”œโ”€โ”€ sounds/             # Sound effects (celebration)
โ”œโ”€โ”€ vendor/
โ”‚   โ”œโ”€โ”€ ffmpeg          # Downloaded ffmpeg binary
โ”‚   โ”œโ”€โ”€ uv              # Downloaded uv binary
โ”‚   โ”œโ”€โ”€ python/         # Python 3.10 installed via uv
โ”‚   โ”œโ”€โ”€ venv/           # Virtual environment with ML packages
โ”‚   โ”œโ”€โ”€ analyzer/       # Extracted analyzer Python scripts
โ”‚   โ””โ”€โ”€ .ready          # Marker indicating setup is complete
โ””โ”€โ”€ models/
    โ”œโ”€โ”€ torch/          # Demucs model cache
    โ”œโ”€โ”€ huggingface/    # WhisperX model cache
    โ””โ”€โ”€ audio_separator/ # UVR Karaoke model cache

Video backgrounds

Pixabay video backgrounds use the Pixabay API. The API key is embedded in release builds. For development, create a .env file at the project root:

PIXABAY_API_KEY=your_key_here

The release script (make-release.sh) sources .env automatically.

Building from source

Prerequisites

Tool Version
Rust 1.85+ (edition 2024)
Linux only libasound2-dev, libudev-dev, libwayland-dev, libxkbcommon-dev

Development build

git clone <repo-url> nightingale
cd nightingale
cargo build --release

Local release

Linux / macOS:

scripts/make-release.sh

Builds the release binary and packages it into nightingale-<target>.tar.gz.

Windows (PowerShell):

powershell -ExecutionPolicy Bypass -File scripts/make-release.ps1

Builds the release binary and packages it into nightingale-x86_64-pc-windows-msvc.zip.

CLI flags

Flag Description
--setup Force re-run of the first-launch bootstrap

Supported platforms

Platform Target
Linux x86_64 x86_64-unknown-linux-gnu
Linux aarch64 aarch64-unknown-linux-gnu
macOS ARM aarch64-apple-darwin
macOS Intel x86_64-apple-darwin
Windows x86_64 x86_64-pc-windows-msvc

License

GPL-3.0-or-later โ€” see LICENSE.