Always get CPUExecutionProvider in immich_machine_learning container

[Triple-Z]

First, thank you for your great photo management app!

Here is my PC:

• i7-4720HQ
• NVIDIA GeForce GTX 970M (Compute Capability 5.2)
• Windows 11
• Ubuntu 22.04 with WSL2
• Installed CUDA 11.8 in WSL2

And my docker-compose file:

version: '3'

services:
  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:release
    volumes:
      - model-cache:/cache
    restart: always
    ports:
      - 3003:3003
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu
                - compute
                - video

volumes:
  model-cache:

However, I always get CPUExecutionProvider in the container log like this following image.

How can I get GPU acceleration for machine learning container? Which log or which test should I look in?

[mertalev]

The image tag should be release-cuda.

[Triple-Z]

I fixed it and restart the container, now the log is:

2024-03-21 02:12:57 [03/20/24 18:12:57] INFO     Setting 'buffalo_l' execution providers to ['CUDAExecutionProvider', 'CPUExecutionProvider'], in descending order of preference

How can I check it really use the GPU?

[mertalev]

You can check GPU usage with nvtop. But it wouldn't show the CUDA provider if it wasn't available and would crash if it couldn't actually use CUDA when loading a model.

[Triple-Z]

Thank you very much, I have solved this problem!

[steve3j]

Bringing this back from the dead as I can't seem to find an answer. How can we completely disable the cpuExecutionProvider. Commented out the cpu in hw.accel.yml, but no good.

I've got a remote server with a weak cpu but a gpu. I'd rather it only use the remote GPU or fallback to the internal cpu. Currently uses the gpu, but ocr is wack and chugs on cpu. It's way slower on the remote machine unless I turn that container off and let it use local cpu.