Wakeword

You can use openwakeword to trigger the chatbot with a wake word. This allows you to interact with the chatbot hands-free, making it more convenient to use in various scenarios.

More details can be found in the openwakeword GitHub repository

Install openwakeword

Whisplay Chatbot use the openwakeword Python package >= 0.6.0, you can install it using pip:

pip install openwakeword --break-system-packages

Note: The official Raspberry Pi OS image may have an older version of openwakeword(0.4.0), also the python version 3.12 is too new for the openwakeword 0.6.0, so you need to setup a virtual environment with python 3.11 to install the compatible openwakeword version:

install pyenv and python 3.11:

sudo apt update
sudo apt install -y build-essential make gcc \
libssl-dev zlib1g-dev libbz2-dev libreadline-dev \
libsqlite3-dev curl llvm libncursesw5-dev xz-utils \
tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev git

curl https://pyenv.run | bash

Add pyenv to ~/.bashrc profile:

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

Install python 3.11 and create a virtual environment:

source ~/.bashrc
pyenv install 3.11.0
pyenv virtualenv 3.11.0 python311
pyenv activate python311

You will find the python path of the virtual environment, it should be something like ~/.pyenv/versions/python311/bin/python, you can later set this path in the .env file to let Whisplay use this python environment to run the openwakeword.

Then you can install the compatible openwakeword version in the virtual environment:

pip install openwakeword
pip install "numpy<2"

download wake word models

You can download the pre-trained wake word models from the openwakeword GitHub repository or use your own custom models. The default wake word is "hey_jarvis", and you can specify multiple wake words if needed.

python -c "import openwakeword.utils as u; u.download_models()"

Configure Whisplay AI Chatbot to use openwakeword

.env file settings:

## Wake word
# enable wake word detection. This feature depends on the openwakeword, you need to install the openwakeword 0.6.0 or above and set up the wake word models correctly for it to work.
WAKE_WORD_ENABLED=true
# if you have setup virtual environment for the wake word python script, you can specify the python binary path here, otherwise it will use "python3" by default
# WAKE_WORD_PYTHON_PATH=/home/pi/.pyenv/versions/python311/bin/python
# Wake word names (comma-separated) or custom model paths (comma-separated)
WAKE_WORDS=hey_jarvis
# WAKE_WORD_MODEL_PATHS=/path/to/custom.tflite
# Detection threshold and cooldown (seconds)
# WAKE_WORD_THRESHOLD=0.5
# WAKE_WORD_COOLDOWN_SEC=1.5
# Auto listening limits (seconds)
# WAKE_WORD_IDLE_TIMEOUT_SEC=60
# WAKE_WORD_RECORD_MAX_SEC=60
# End conversation keywords (comma-separated, case-insensitive)
# WAKE_WORD_END_KEYWORDS=byebye,goodbye,stop

Restart the chatbot after updating the .env file, and it will start listening for the specified wake words. When a wake word is detected, the chatbot will become active and ready to respond to your commands or queries. You can adjust the detection threshold, cooldown period, and other parameters by modifying the corresponding environment variables in the .env file.

Note

Once the device is awake, it will play a chime sound to indicate that it is ready to listen. The session will automatically end after a period of inactivity (default is 60 seconds). If voice activity is detected, the session will continue until there is a period of silence (default is 1.5 seconds) or if an end conversation keyword is detected in the user's speech.

The auto listening will start again after the llm answer is finished, you can end the conversation by saying the end conversation keywords (e.g. "byebye", "goodbye", "stop"), or just let it idle until the idle timeout is reached. If end word is detected, the chatbot will sleep after the current response is finished.