Can't start via docker-compose on RTX 5080

[ekzotech]

I'm using WSL distro from Nvidia Workbench which has access to my GPU.
After docker-compose up -d I went to a http://localhost:8000 and saw this error message:
Failed to load model: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

I've updated Dockerfile and replaced index-url for torch with version https://download.pytorch.org/whl/nightly/cu128 (which works with 5080). I've removed the image and rebuilt it still no luck and the same error message.

[asabino2]

Same error here with a RTX 5090

[fspecii]

RTX 5080 (Blackwell Architecture) Support

The RTX 5080 uses NVIDIA's new Blackwell architecture (SM 10.0) which requires:

• CUDA 12.8+ (current Docker image uses CUDA 12.1)
• PyTorch nightly build with cu128 support

Workaround - Build Custom Image

1. Clone the repo and modify Dockerfile:

# Change line 62 from:
RUN pip3 install --no-cache-dir torch --index-url https://download.pytorch.org/whl/cu121

# To:
RUN pip3 install --no-cache-dir --pre torch --index-url https://download.pytorch.org/whl/nightly/cu128

2. Also update torchvision/torchaudio (line 65):

RUN pip3 install --no-cache-dir --pre torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

3. Update base image to CUDA 12.8 (line 26):

FROM nvidia/cuda:12.8.0-cudnn9-runtime-ubuntu22.04

4. Build locally:

docker compose up --build

Note

PyTorch nightly builds can be unstable. Once PyTorch officially releases cu128 support (expected in PyTorch 2.6), we'll update the official Docker image.

For now, RTX 50 series users need to build from source with the modifications above.

[agrrh]

Could you please double check, which image to use?

https://hub.docker.com/r/nvidia/cuda/tags?name=cudnn9-runtime-ubuntu

$ docker pull nvidia/cuda:12.8.0-cudnn9-runtime-ubuntu22.04
Error response from daemon: manifest for nvidia/cuda:12.8.0-cudnn9-runtime-ubuntu22.04 not found: manifest unknown: manifest unknown

[ekzotech]

Could you please double check, which image to use?

https://hub.docker.com/r/nvidia/cuda/tags?name=cudnn9-runtime-ubuntu

$ docker pull nvidia/cuda:12.8.0-cudnn9-runtime-ubuntu22.04
Error response from daemon: manifest for nvidia/cuda:12.8.0-cudnn9-runtime-ubuntu22.04 not found: manifest unknown: manifest unknown

I believe it should be nvidia/cuda:12.8.0-cudnn-runtime-ubuntu22.04, because this image exists in the docker hub.

[asabino2]

Hi,
I modified the Dockerfile, but the error persists.

[Naman-Dharmani]

Hi
Yes the error persists due to line 12 in backend/requirements.txt which attempts to override with torch v2.4.1 as hardcoded in heartlib's pyproject.toml.

Not the best workaround, but updating Dockerfile to install requirements.txt first and then overriding it with torch version compatible with Blackwell (cu128+) worked for me.

Updated Dockerfile

# ...existing code

# Install Python dependencies in separate layers for smaller Docker Hub uploads
# Layer 1: pip upgrade
RUN pip3 install --no-cache-dir --upgrade pip

# Layer 2: Other requirements
RUN pip3 install --no-cache-dir -r /app/backend/requirements.txt

# Layer 3: PyTorch (largest dependency ~800MB)
# RUN pip3 install --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
RUN pip3 install --no-cache-dir --pre --force-reinstall torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu130

# Layer 4: bitsandbytes and accelerate
RUN pip3 install --no-cache-dir bitsandbytes accelerate

Note: I have merged torch and torchaudio/torchvision install into a single layer, you can keep it separate as it is now. Also, cu128+ builds are stable now, so need not use nightly builds in the index url.

A better workaround would be clone the heartlib repo into a tmp folder, remove hardcoded torch versions and then pip3 install using it (after removing line 12 from requirements.txt), to keep the current layers intact for better caching. It saves duplicate torch installs too.

I feel project should shift to using uv package manager for faster pip installs in docker build.

[asabino2]

Run, but now is displaying this message

[Naman-Dharmani]

Yes, for that, as discussed in heartlib's issue 75 here, need to replace torchcodec.save with soundfile backend for saving the audio file, that solves the issue.

Note: music_generation.py is named as music_service.py file in this studio project. Need to replace at line 704

[fspecii]

Update: Soundfile Fallback Added

Just pushed a fix that adds automatic soundfile fallback for RTX 50 series (commit ffb63fe).

Now the code will:

1. Try torchaudio.save first
2. If it fails (Blackwell GPUs), automatically fallback to soundfile

Complete RTX 5080/5090 Setup

For RTX 50 series users, you still need to build with CUDA 12.8+ and PyTorch cu128+. Here's the updated Dockerfile changes:

1. Update base image (line 26):

FROM nvidia/cuda:12.8.0-cudnn-runtime-ubuntu22.04

2. Update PyTorch install - install requirements first, then override PyTorch:

# Install requirements first
RUN pip3 install --no-cache-dir -r /app/backend/requirements.txt

# Override with cu128+ PyTorch (stable, not nightly)
RUN pip3 install --no-cache-dir --force-reinstall torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

3. Build locally:

docker compose build --no-cache
docker compose up

The soundfile fallback is now in the main branch, so once you build with the updated Dockerfile, the audio saving should work automatically.

Thanks @Naman-Dharmani and @aweworks for the debugging help! 🙏

[Artem-Glebov]

Same issue RTX 5070.
Did 2. Update PyTorch install - install requirements first, then override PyTorch as comment bellow
Then docker compose build --no-cache && docker compose up -d