Hardware encoding [information-not-issue]

[Fluxed1]

The bug

hardware acceleration on Nvidia grid vgpu not functioning with included docker-compose.yml, had to change /etc/docker/daemon.json and docker compose yaml

Im using a gtx1650 with vgpu unlock

Nvidia-smi does not funtion under docker container and does not use gpu

Failed to initialize NVML: Unknown Error

The OS that Immich Server is running on

Ubuntu 20.04

Version of Immich Server

v1.101.0

Version of Immich Mobile App

n/a

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ['start.sh', 'immich']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
              - capabilities: [gpu]
    command: ['start.sh', 'microservices']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
              - capabilities: [gpu]
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:51d6c56749a4243096327e3fb964a48ed92254357108449cb6e23999c37773c5
    restart: always

  database:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis
LOG_LEVEL=debug

Reproduction steps

Have a Fake vgpu apparently or something.

Additional information

fixes:

• set
  no-cgroups = false

in /etc/nvidia-container-runtime/config.toml

• set
  /etc/docker/daemon.json
  {
  "default-runtime": "nvidia",
  "exec-opts": [
  "native.cgroupdriver=cgroupfs"
  ],
  "runtimes": {
  "nvidia": {
  "args": [],
  "path": "nvidia-container-runtime"
  }
  }
  }

and change in docker
deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: - gpu - compute - video

TO

runtime: nvidia deploy: resources: reservations: devices: - capabilities: [gpu]

[pandaedward]

confirmed in hwaccel.ml.yml I need to added runtime: nvidia to passthrough the gpu to the machine-learning container.

  cuda:
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu