[Detector Support]: AMD NPU

[azharkov78]

Describe the problem you are having

Hi
Can i use AMD NPU in frigate? In host:

• i have kernel 6.14.0-24
• Install https://github.com/amd/xdna-driver#
• i see /dev/accel/accel0
  -Processor information AMD Ryzen 7 8845HS w/ Radeon 780M Graphics, 16 cores
  In frigate i see /dev/accel/accel0 and all /dev/dri

Version

0.16.0-3eb3797

Frigate config file

detectors:
  onnx_0:
    type: onnx
  openvino_0: >> it's for testing
    type: openvino
    device: CPU 
model:
  path: plus:

docker-compose file or Docker CLI command

frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:3eb3797-rocm
    cap_add:
      - CAP_PERFMON
    shm_size: "2048mb" # update for your cameras based on calculation above
    devices:
      #- /dev/dri/renderD128:/dev/dri/renderD128 # For intel hwaccel, needs to be updated for your hardware
      #- /dev/dri/card0:/dev/dri/card0
      - /dev/dri
      - /dev/kfd
      - /dev/video0:/dev/video0
      - /dev/accel/
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - videos_frigate:/media/frigate
      - /opt/frigate/config:/config
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 3000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
      - "1984:1984/tcp"
      - "8971:8971/tcp"
    environment:
      LIBVA_DRIVER_NAME: radeonsi
      PLUS_API_KEY: xxxxxx
      HSA_OVERRIDE_GFX_VERSION: "11.0.0"

Relevant Frigate log output

No Relevant Frigate log

Install method

Docker Compose

Object Detector

Other

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

Inference Speed is good in yolonas 320 (But in yolov9 better, but the recognition of small objects is worse than in yolonas )

[hawkeye217]

No, Frigate has no specific code to take advantage of AMD NPUs at this time. Your setup is currently using the AMD GPU successfully, though.

[azharkov78]

no specific code for AMD only or also Intel?
I'm thinking about changing to Core Ultra 5 125H

[hawkeye217]

Both the iGPU and NPU from Intel are supported, but NPU requires a bit more work. Both perform similarly, so we still recommend the iGPU at this time.

See #13248 (comment)

[NickM-27]

NPU for Intel performs poorly for models other than very small models so it isn't worth the effort of implementation yet

[akostadinov]

I see Intel NPU support was added in #20536 so does it work fast and power efficient or this comment still stands correct?

I'm still looking at the options for a low-power detection support and came to this thread.

P.S. also I wonder if there were any plan or experiments with Ryzen NPUs now they have linux support.

[NickM-27]

No it is no longer correct, Intel NPUs work quite well. AMD NPUs are currently at this point, they don't work well and are slow for these object detection models. Seems unlikely to change given the focus on LLMs