Skip to content
BranchesTags

Repository files navigation

Winterfresh 🌨️

A fast, concise voice assistant for Raspberry Pi with wake word detection, conversation memory, and barge-in support.

Features

  • 🎤 Local wake word detection ("winterfresh") - no API cost for listening
  • 💬 Conversational memory (5-minute timeout)
  • 🔇 Barge-in support (interrupt while speaking)
  • 🎵 Audio feedback with elegant chimes
  • 🔄 Auto-restart on inactivity
  • 🎙️ Hardware echo cancellation (with USB speakerphone)

Hardware Requirements

  • Raspberry Pi 5 (8GB recommended) or Raspberry Pi 4 (4GB+)
  • Raspberry Pi 5 27W USB-C Power Supply (recommended for Pi 5 with USB peripherals)
  • SanDisk 64GB Extreme microSDXC UHS-I Memory Card (fast boot and read/write)
  • USB Speakerphone with echo cancellation (recommended):
    • EMEET Conference Speakerphone M0 Plus

Prerequisites

  • Node.js v20+
  • Python 3.11+
  • OpenAI API key
  • Sox audio tools

Installation

1. System Setup (Raspberry Pi)

# Update system
sudo apt-get update
sudo apt-get upgrade -y

# Install Node.js v20
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs

# Install Python 3.11+ and pip
sudo apt-get install -y python3 python3-pip python3-venv

# Install sox for audio recording/playback
sudo apt-get install -y sox libsox-fmt-all

# Install portaudio (required for sounddevice)
sudo apt-get install -y portaudio19-dev

# Install git
sudo apt-get install -y git

# Verify versions
node --version    # Should be v20+
python3 --version # Should be 3.11+

2. Clone & Setup Project

# Clone repository
git clone https://github.com/xiaolin/winterfresh.git ~/winterfresh
cd ~/winterfresh

# Install Node.js dependencies
npm install

# Build TypeScript
npm run build

3. Python Virtual Environment Setup

# Create virtual environment
python -m venv .venv

# Activate it
source .venv/bin/activate

# Install Python dependencies
pip install -r requirements.txt

# Verify installation
python -c "import numpy, sounddevice, vosk; print('✅ All Python packages installed')"

# Deactivate (optional, the app will use .venv/bin/python directly)
deactivate

4. Download Vosk Model (for local wake word detection)

# Create models directory
mkdir -p models
cd models

# Download small English model (~50MB)
wget https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip
unzip vosk-model-small-en-us-0.15.zip
rm vosk-model-small-en-us-0.15.zip

cd ..

5. Environment Configuration

# Create environment file
vi .env

Environment variables in .env:

Variable Default Description
OPENAI_API_KEY (required) Your OpenAI API key
CHAT_MODEL gpt-4o-mini OpenAI chat model
TRANSCRIBE_MODEL gpt-4o-mini-transcribe OpenAI transcription model
TTS_MODEL gpt-4o-mini-tts OpenAI text-to-speech model
WINTERFRESH_MAX_TURNS 20 Max conversation turns before trimming
SAMPLE_RATE 24000 Audio sample rate for recording
ARECORD_DEVICE mic_share ALSA device for recording (Linux)
ARECORD_CHANNELS 2 Audio channels for recording (Linux)
ALSA_PLAY_DEVICE plughw:2,0 ALSA device for playback (Linux)
ALSA_CARD 2 ALSA mixer card for volume (Linux)

Example .env file:

OPENAI_API_KEY=your_openai_api_key_here
CHAT_MODEL=gpt-4o-mini
TRANSCRIBE_MODEL=gpt-4o-mini-transcribe
TTS_MODEL=gpt-4o-mini-tts
WINTERFRESH_MAX_TURNS=20
ARECORD_DEVICE=mic_share
ARECORD_CHANNELS=2
ALSA_PLAY_DEVICE=plughw:CARD=Plus,DEV=0
ALSA_CARD=Plus

6. Audio Device Setup (Raspberry Pi)

6.1 Check USB Audio Device

# Plug in your USB speakerphone

# Check if detected
cat /proc/asound/cards
# Should show your USB device (e.g., card 2: EMEET OfficeCore M0 Plus)

arecord -l  # List capture devices
aplay -l    # List playback devices

6.2 Configure ALSA for Shared Microphone Access

Create a system-wide ALSA config so the mic_share recording device exists after reboot and is visible to PM2. First confirm the stable card id:

cat /proc/asound/cards
arecord -l

For the EMEET OfficeCore M0 Plus, the card often appears like this:

2 [Plus           ]: USB-Audio - EMEET OfficeCore M0 Plus
card 2: Plus [EMEET OfficeCore M0 Plus], device 0: USB Audio [USB Audio]

Use the stable card id, Plus in this example, not the numeric card hw:2,0. The numeric card can change after reboot.

sudo nano /etc/asound.conf

Add this config, replacing Plus if your card id is different:

pcm.mic_share {
    type dsnoop
    ipc_key 12345
    ipc_perm 0666
    slave {
        pcm "hw:CARD=Plus,DEV=0"
        rate 16000
        channels 2
    }
}

Then verify that ALSA can see the shared microphone:

arecord -D mic_share -f S16_LE -c 2 -r 16000 -d 3 /tmp/test.wav
aplay -D plughw:CARD=Plus,DEV=0 /tmp/test.wav

If you prefer a project-local config, copy config/asoundrc.example to config/asoundrc and update the CARD value there. The PM2 config loads it through ALSA_CONFIG_PATH when that file exists. The system-wide /etc/asound.conf path is simpler and more reliable on boot.

6.3 Configure PulseAudio

# Check PulseAudio is running
systemctl --user status pulseaudio

# If not running, start it
systemctl --user start pulseaudio

# List available sinks (output) and sources (input)
pactl list sinks short
pactl list sources short

# Set your USB device as default (use actual device names from above)
# Example for EMEET speakerphone:
pactl set-default-sink alsa_output.usb-EMEET_EMEET_OfficeCore_M0_Plus_<YOUR_DEVICE_ID>.analog-stereo
pactl set-default-source alsa_input.usb-EMEET_EMEET_OfficeCore_M0_Plus_<YOUR_DEVICE_ID>.analog-stereo

6.4 Make PulseAudio Settings Persistent

mkdir -p ~/.config/pulse
cat > ~/.config/pulse/default.pa << 'EOF'
.include /etc/pulse/default.pa
set-default-sink alsa_output.usb-EMEET_EMEET_OfficeCore_M0_Plus_<YOUR_DEVICE_ID>.analog-stereo
set-default-source alsa_input.usb-EMEET_EMEET_OfficeCore_M0_Plus_<YOUR_DEVICE_ID>.analog-stereo
EOF

Note: Replace the device names with your actual device names from pactl list sinks short and pactl list sources short.

6.5 Test Audio

# Test recording with shared device
arecord -D mic_share -f S16_LE -c 2 -r 16000 -d 3 /tmp/test.wav

# Test playback with the same stable card id
aplay -D plughw:CARD=Plus,DEV=0 /tmp/test.wav

# Test simultaneous recording (both should work without "Device busy" error)
arecord -D mic_share -f S16_LE -c 2 -r 16000 -d 10 /tmp/test1.wav &
arecord -D mic_share -f S16_LE -c 2 -r 16000 -d 5 /tmp/test2.wav

# Test Python audio
source .venv/bin/activate
python -c "import sounddevice as sd; print(sd.query_devices())"
deactivate

7. Test the Setup

# Test wake word detection first
source .venv/bin/activate
python wake.py
# Say "winterfresh" - should print "WAKE" and exit
deactivate

# Test full app
npm run dev
# Or: npx tsx src/app.ts

8. Install PM2 for Production

# Install PM2 globally
sudo npm install -g pm2

# Build TypeScript
npm run build

# Start winterfresh using ecosystem config
pm2 start pm2.config.cjs

# Or start directly
pm2 start dist/app.js --name winterfresh

# Save PM2 configuration
pm2 save

# Setup auto-start on boot
pm2 startup
# Follow the command it outputs (copy/paste and run it)

# Verify it's running
pm2 status
pm2 logs winterfresh --lines 50

Usage

Voice Commands

  1. Wake the assistant: Say "winterfresh", "winter fresh", or "hey winterfresh"
  2. Speak your request: After hearing the wake chime
  3. Interrupt anytime: Start speaking to interrupt long responses
  4. Wait for timeout: 7 seconds of inactivity returns to wake word mode

Audio Feedback

  • 🎵 Wake chime - Wake word detected, ready to listen
  • 🎵 Processing chime - Processing your request, stops when assistant speaks

PM2 Commands

# View live logs
pm2 logs winterfresh

# Monitor resource usage
pm2 monit

# Restart after code changes
pm2 restart winterfresh

# Stop the assistant
pm2 stop winterfresh

# Start the assistant
pm2 start winterfresh

Development

Local Testing (Mac)

Note: For Mac testing, use headphones to avoid echo (built-in speakers will be heard by built-in mic).

# Install system dependencies (Mac)
brew install sox portaudio

# Setup Python venv
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
deactivate

# Download Vosk model
mkdir -p models && cd models
curl -LO https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip
unzip vosk-model-small-en-us-0.15.zip
cd ..

# Install Node dependencies
npm install

# Run
npm run dev

Update Script

Create update.sh for easy updates:

#!/bin/bash
cd ~/winterfresh
git pull
npm install
npm run build
source .venv/bin/activate
pip install -r requirements.txt 2>/dev/null || pip install numpy sounddevice vosk
deactivate
pm2 restart winterfresh
echo "✅ Winterfresh updated!"

Make it executable:

chmod +x update.sh

Troubleshooting

Wake word not detecting

# Test Python script directly
source .venv/bin/activate
python wake.py
# Speak and watch the audio level bar - it should move
# Say "winterfresh" clearly
deactivate

# If no audio level, check your mic:
python -c "import sounddevice as sd; print(sd.query_devices())"

"No module named numpy" error

# Make sure packages are in the venv, not global Python
.venv/bin/pip install numpy sounddevice vosk

# Verify
.venv/bin/python -c "import numpy, sounddevice, vosk; print('OK')"

Audio device not found

# List available devices
arecord -l
aplay -l

# Check Python sees the device
source .venv/bin/activate
python -c "import sounddevice as sd; print(sd.query_devices())"

Unknown PCM mic_share after reboot

This means the USB speakerphone may be detected, but the ALSA alias mic_share is not defined or not visible to the process.

# Confirm the hardware exists
lsusb
cat /proc/asound/cards
arecord -l
aplay -l

# Check whether mic_share is configured
cat ~/.asoundrc
cat /etc/asound.conf
arecord -L | grep -A5 mic_share

If both ~/.asoundrc and /etc/asound.conf are missing, create /etc/asound.conf using the setup in section 6.2. For the current EMEET setup, the important line is:

pcm "hw:CARD=Plus,DEV=0"

After creating it:

arecord -D mic_share -f S16_LE -c 2 -r 16000 -d 3 /tmp/test.wav
pm2 restart winterfresh-realtime --update-env

PulseAudio connection refused

# Check if PulseAudio is running
systemctl --user status pulseaudio

# Restart PulseAudio
systemctl --user restart pulseaudio

# Wait a moment then test
sleep 2
pactl info

PulseAudio sink missing after reboot

# Check current sinks
pactl list sinks short

# If your USB device isn't the default, set it again
pactl set-default-sink alsa_output.usb-EMEET_EMEET_OfficeCore_M0_Plus_<YOUR_DEVICE_ID>.analog-stereo

# Make sure ~/.config/pulse/default.pa exists with correct settings
cat ~/.config/pulse/default.pa

"Device or resource busy" error

This happens when multiple processes try to access the microphone without using the shared dsnoop device.

# Verify the system-wide ALSA config exists
cat /etc/asound.conf

# If missing, create it using the stable card id from arecord -l
arecord -l

# Make sure .env uses mic_share
grep ARECORD .env
# Should show: ARECORD_DEVICE=mic_share

No audio output / "no default audio device"

# Check PulseAudio sink
pactl list sinks short
pactl get-default-sink

# If shows auto_null or missing, your USB device isn't detected
# Replug the USB device and restart PulseAudio
systemctl --user restart pulseaudio

# Verify your USB device is now the default
pactl list sinks short
pactl set-default-sink alsa_output.usb-EMEET_EMEET_OfficeCore_M0_Plus_<some_long_string>

Permission issues (Raspberry Pi)

# Add user to audio group
sudo usermod -a -G audio $USER

# Logout and login again

Mac microphone permission

Go to System Settings → Privacy & Security → Microphone and enable for:

  • Terminal (or iTerm)
  • Visual Studio Code (if running from VS Code)

Quit and reopen the app after enabling.

Performance

  • Wake word detection: Local (Vosk) - no API cost
  • Speech-to-text: OpenAI Whisper API
  • Chat: OpenAI GPT-4o-mini
  • Text-to-speech: OpenAI TTS

Raspberry Pi 5 (8GB):

  • Memory usage: ~150-200MB
  • CPU usage: ~5-10% during wake word listening
  • Wake word latency: ~0.3-0.5 second
  • Response latency: ~1-2 seconds (API dependent)

Raspberry Pi 4 (4GB):

  • Memory usage: ~150-200MB
  • CPU usage: ~10-20% during wake word listening
  • Wake word latency: ~0.5-1 second
  • Response latency: ~1-3 seconds (API dependent)

License

MIT

About

AI assistant on Rasberry Pi

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages