TrueNAS Cuda Issue

[GinSoakedBoy]

I'm getting the following error in Scriberr when I try to run a transcription using Cuda: Unexpected error from cudaGetDeviceCount()

I'm running Sciberr within a docker container from Portainer on a TrueNAS server. My GPU is a GTX 1060 (3GB).

Below are the driver versions for TrueNAS and Scriberr.

TrueNAS:
NVIDIA-SMI 550.142
Driver Version: 550.142
CUDA Version: 12.4

Scriberr docker container:
NVIDIA-SMI 550.142
Driver Version: 550.142
CUDA Version: 12.6

Here's my docker compose file:

services:
  scriberr:
    image: ghcr.io/rishikanthc/scriberr-cuda:latest
    container_name: scriberr
    environment:
      - PUID=568
      - PGID=568
      - TZ=America/New_York
      - APP_ENV=production
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
    volumes:
      - /mnt/#######/scriberr/data:/app/data
      - /mnt/#######/scriberr/env_data:/app/whisperx-env 
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    restart: unless-stopped
    labels: 
    networks:
      - #####

networks:
  ######:
    external: true

I can't update the Cuda version in TrueNAS as it's tied to OS updates, is there a way to force a lower version of Cuda in Scriberr? Or is there another way to fix this?

[rishikanthc]

To use a lower version of Cuda, the Scriberr image has to be built against a lower cuda image.. Basically will have to compile a new image with cuda 12.4 as the base image..
While I look into this, can you try the following?

The most common fix for cudaGetDeviceCount() errors in PyTorch is to force it to use a different detection method. Add the following environment variables to your docker-compose.yml:

services:
  scriberr:
    image: ghcr.io/rishikanthc/scriberr-cuda:latest
    container_name: scriberr
    runtime: nvidia # <--- Add this
    environment:
      - PUID=568
      - PGID=568
      - TZ=America/New_York
      - APP_ENV=production
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=all  # Changed from compute,utility to all
      - PYTORCH_NVML_BASED_CUDA_CHECK=1 # <--- Add this
      - CUDA_DEVICE_ORDER=PCI_BUS_ID    # <--- Add this

Can you try the above changes and get back to me?