Skip to content

Latest commit

 

History

History
 
 

speaches

README.md

speaches

CONTAINERS IMAGES RUN BUILD

Speaches

Speaches supports VAD (Voice Activity Detection), STT (Speech-to-Text) and TTS (Text-to-Speech).

What is VAD and why it's needed

Voice Activity Detection (VAD) is used to detect the presence or absence of human speech in audio streams. It is important when processing speech because by identifying when someone is actually speaking, you prevent unnecessary processing of silence or background noise, reducing computational overhead.

Important Notes on TTS Implementation

At this time of adding support to Speaches, Kokoro uses onnxruntime and kokoro-onnx for TTS.

This backend is currently slower than kokoro-tts:fastapi, which also implements the OpenAI protocol, and can be used separately.

Build

CUDA_VERSION=12.9 LSB_RELEASE=24.04 PYTHON_VERSION=3.12 jc build speaches

or

CUDA_VERSION=12.9 LSB_RELEASE=24.04 PYTHON_VERSION=3.12  jetson-containers build --build-args=PIP_RETRIES:10,PIP_TIMEOUT:60 speaches

Run

Run it with:

docker run -it --rm --network=host <docker-image-name>

Speaches site is at:

http://localhost:8000/

Contributing

If you need TTS endpoint support, please open an issue and tag @OriNachum

CONTAINERS
speaches
   Requires L4T ['>=34.1.0']
   Dependencies build-essential pip_cache:cu126 cuda:12.6 cudnn:9.3 python cmake ctranslate2 tensorrt numpy onnx onnxruntime huggingface_hub faster-whisper pytorch:2.8 torchaudio espeak piper-tts numba kokoro-tts:onnx nodejs
   Dockerfile Dockerfile
   Images dustynv/speaches:r36.4.0-cu128-24.04 (2025-03-03, 6.6GB)
CONTAINER IMAGES
Repository/Tag Date Arch Size
  dustynv/speaches:r36.4.0-cu128-24.04 2025-03-03 arm64 6.6GB

Container images are compatible with other minor versions of JetPack/L4T:
    • L4T R32.7 containers can run on other versions of L4T R32.7 (JetPack 4.6+)
    • L4T R35.x containers can run on other versions of L4T R35.x (JetPack 5.1+)

RUN CONTAINER

To start the container, you can use jetson-containers run and autotag, or manually put together a docker run command:

# automatically pull or build a compatible container image
jetson-containers run $(autotag speaches)

# or explicitly specify one of the container images above
jetson-containers run dustynv/speaches:r36.4.0-cu128-24.04

# or if using 'docker run' (specify image and mounts/ect)
sudo docker run --runtime nvidia -it --rm --network=host dustynv/speaches:r36.4.0-cu128-24.04

jetson-containers run forwards arguments to docker run with some defaults added (like --runtime nvidia, mounts a /data cache, and detects devices)
autotag finds a container image that's compatible with your version of JetPack/L4T - either locally, pulled from a registry, or by building it.

To mount your own directories into the container, use the -v or --volume flags:

jetson-containers run -v /path/on/host:/path/in/container $(autotag speaches)

To launch the container running a command, as opposed to an interactive shell:

jetson-containers run $(autotag speaches) my_app --abc xyz

You can pass any options to it that you would to docker run, and it'll print out the full command that it constructs before executing it.

BUILD CONTAINER

If you use autotag as shown above, it'll ask to build the container for you if needed. To manually build it, first do the system setup, then run:

jetson-containers build speaches

The dependencies from above will be built into the container, and it'll be tested during. Run it with --help for build options.