When using immich 2.4.1 and calling NPU from within Docker, the logs show dynamic input, but my research indicates that rknn only supports static input. How can I modify this?

[YKN-hub]

I have searched the existing issues, both open and closed, to make sure this is not a duplicate report.

• Yes

The bug

Is anyone there? I have a small problem and need help. I deployed Immich on a RK3566 box and want to enhance machine learning by utilizing the RK3566's NPU. My box's NPU driver is version 0.9.8. When I view the call logs of the image pulled from Docker, it always shows RKNN-Toolkit2 version 2.3.0, which is incompatible. I upgraded it to 2.3.2 in Docker, but it still doesn't display correctly. The face recognition and detection model I'm using is Antelopev2, which I downloaded separately from https://huggingface.co/immich-app.

The OS that Immich Server is running on

armbian-bullseye

Version of Immich Server

2.4.1

Version of Immich Mobile App

2.4.1

Platform with the issue

• Server
• Web
• Mobile

Device make and model

rk3566

Your docker-compose.yml content

#
# WARNING: To install Immich, follow our guide: https://docs.immich.app/install/docker-compose
#
# Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/data
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - '2283:2283'
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-rknn
    extends:
      file: hwaccel.ml.yml
      service: rknn
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    environment:
      - MACHINE_LEARNING_RKNN_THREADS=2  # 对于RK3566，建议尝试设置为2
      - MACHINE_LEARNING_CLIP_ENABLED=true
      - MACHINE_LEARNING_FACIAL_RECOGNITION_ENABLED=true
      - MACHINE_LEARNING_OCR_ENABLED=true
      # 离线模式环境变量
      - TRANSFORMERS_OFFLINE=1
      - HF_HUB_OFFLINE=1
      - TRANSFORMERS_OFFLINE=1
      - HF_DATASETS_OFFLINE=1
      - HUGGINGFACE_HUB_OFFLINE=1
      # 指定使用本地模型
      - IMMICH_MACHINE_LEARNING_OFFLINE=true
      # 禁用自动下载和更新
      - DISABLE_AUTO_DOWNLOAD=true
      - DISABLE_MODEL_UPDATES=true
      # 如果模型加载失败，跳过而不是重试下载
      - MACHINE_LEARNING_SKIP_MODEL_VALIDATION=true
    restart: always
    healthcheck:
      disable: false
      # 修改健康检查，因为离线模式可能无法加载完整模型
      test: ["CMD-SHELL", "pgrep -f 'python.*machine_learning' || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 120s

  redis:
    container_name: immich_redis
    image: docker.io/valkey/valkey:8@sha256:81db6d39e1bba3b3ff32bd3a1b19a6d69690f94a3954ec131277b9a26b95b3aa
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
      # Uncomment the DB_STORAGE_TYPE: 'HDD' var if your database isn't stored on SSDs
      # DB_STORAGE_TYPE: 'HDD'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    shm_size: 128mb
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://docs.immich.app/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/immich/uploads

# The location where your database files are stored. Network shares are not supported for the database
DB_DATA_LOCATION=/mnt/immich/database

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=Asia/Shanghai

# The Immich version to use. You can pin this to a specific version like "v2.1.0"
IMMICH_VERSION=v2

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=ZTFkNzVhZm

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

Is anyone there? I have a small problem and need help. I deployed Immich on a RK3566 box and want to enhance machine learning by utilizing the RK3566's NPU. My box's NPU driver is version 0.9.8. When I view the call logs of the image pulled from Docker, it always shows RKNN-Toolkit2 version 2.3.0, which is incompatible. I upgraded it to 2.3.2 in Docker, but it still doesn't display correctly. The face recognition and detection model I'm using is Antelopev2, which I downloaded separately from https://huggingface.co/immich-app.

Relevant log output

[01/03/26 22:34:39] INFO     Starting gunicorn 23.0.0                           
[01/03/26 22:34:39] INFO     Listening at: http://[::]:3003 (18)                
[01/03/26 22:34:39] INFO     Using worker: immich_ml.config.CustomUvicornWorker 
[01/03/26 22:34:39] INFO     Booting worker with pid: 28                        
2026-01-03 22:34:41.421167103 [W:onnxruntime:Default, device_discovery.cc:164 DiscoverDevicesForPlatform] GPU device discovery failed: device_discovery.cc:89 ReadFileContents Failed to open file: "/sys/class/drm/card0/device/vendor"
[01/03/26 22:35:00] I        Started server process [28]                        
[01/03/26 22:35:00] I        Waiting for application startup.                   
[01/03/26 22:35:00] I        Created in-memory cache with unloading after 300s  
                             of inactivity.                                     
[01/03/26 22:35:00] I        Initialized request thread pool with 4 threads.    
[01/03/26 22:35:00] I        Application startup complete.                      
[01/03/26 22:35:38] I        Loading visual model 'ViT-L-16-SigLIP2-256__webli' 
                             to memory                                          
[01/03/26 22:35:38] I        Loading RKNN model from                            
                             /cache/clip/ViT-L-16-SigLIP2-256__webli/visual/rknp
                             u/rk3566/model.rknn with 2 threads.                
I RKNN: [22:35:41.449] RKNN Runtime Information, librknnrt version: 2.3.0 (c949ad889d@2024-11-07T11:35:33)
I RKNN: [22:35:41.450] RKNN Driver Information, version: 0.9.8
I RKNN: [22:35:41.459] RKNN Model Information, version: 6, toolkit version: 2.3.0(compiler version: 2.3.0 (c949ad889d@2024-11-07T11:39:30)), target: RKNPU lite, target platform: rk3566, framework name: ONNX, framework layout: NCHW, model inference type: static_shape
W RKNN: [22:35:43.920] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
I RKNN: [22:35:50.976] RKNN Runtime Information, librknnrt version: 2.3.0 (c949ad889d@2024-11-07T11:35:33)
I RKNN: [22:35:50.976] RKNN Driver Information, version: 0.9.8
I RKNN: [22:35:50.980] RKNN Model Information, version: 6, toolkit version: 2.3.0(compiler version: 2.3.0 (c949ad889d@2024-11-07T11:39:30)), target: RKNPU lite, target platform: rk3566, framework name: ONNX, framework layout: NCHW, model inference type: static_shape
W RKNN: [22:36:05.411] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
[01/03/26 22:36:05] I        Loaded RKNN model from                             
                             /cache/clip/ViT-L-16-SigLIP2-256__webli/visual/rknp
                             u/rk3566/model.rknn with 2 threads.                
E RKNN: [22:36:20.063] failed to submit, op id: 1, op name: Conv:/visual/trunk/patch_embed/proj/Conv_split, flags: 0x5, task start: 0, task number: 354, run task counter: 0, int status: 0, If using rknn, update to the latest toolkit2 and runtime from: https://console.zbox.filez.com/l/I00fc3 (PWD: rknn). If using rknn-llm, update from: https://github.com/airockchip/rknn-llm
E RKNN: [22:36:20.170] failed to submit, op id: 1, op name: Conv:/visual/trunk/patch_embed/proj/Conv_split, flags: 0x5, task start: 0, task number: 354, run task counter: 0, int status: 0, If using rknn, update to the latest toolkit2 and runtime from: https://console.zbox.filez.com/l/I00fc3 (PWD: rknn). If using rknn-llm, update from: https://github.com/airockchip/rknn-llm
E RKNN: [22:36:32.223] failed to submit, op id: 1, op name: Conv:/visual/trunk/patch_embed/proj/Conv_split, flags: 0x5, task start: 0, task number: 354, run task counter: 0, int status: 0, If using rknn, update to the latest toolkit2 and runtime from: https://console.zbox.filez.com/l/I00fc3 (PWD: rknn). If using rknn-llm, update from: https://github.com/airockchip/rknn-llm
E RKNN: [22:36:32.331] failed to submit, op id: 1, op name: Conv:/visual/trunk/patch_embed/proj/Conv_split, flags: 0x5, task start: 0, task number: 354, run task counter: 0, int status: 0, If using rknn, update to the latest toolkit2 and runtime from: https://console.zbox.filez.com/l/I00fc3 (PWD: rknn). If using rknn-llm, update from: https://github.com/airockchip/rknn-llm

Additional information

No response

[tribhuwan-kumar]

$ sudo podman logs -f immich_ml
doaspassword:
[03/03/26 07:45:40] INFO     Starting gunicorn 23.0.0
[03/03/26 07:45:40] INFO     Listening at: http://[::]:3003 (10)
[03/03/26 07:45:40] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[03/03/26 07:45:40] INFO     Booting worker with pid: 18
2026-03-03 07:45:41.574399717 [W:onnxruntime:Default, device_discovery.cc:164 DiscoverDevicesForPlatform] GPU device discovery failed: device_discovery.cc:89 ReadFileContents Failed to open file: "/sys/class/drm/card0/device/vendor"

same issue GPU device discovery failed: