AMD NPU

[mikealanni]

I saw this Dragon-NPU that support AMD NPU on linux, and I would like to know if this can integrated with Frigate to use the NPU.

Thanks in advance

[NickM-27]

Looks promising, but AMD NPU support is very new and Intel NPU can be supported directly with OpenVINO so there isn't much need for that.

It would be interesting to see if ROCm can use it directly or not. The main problem with adding support is that we don't have any devices with NPUs built in to test with

[mikealanni]

I can help to help your project as it is awesome :)

[bagobones]

Something to watch is that AMD only has limited NPU support in there own lemonade project, no Linux NPU support.

So if you want to watch for developments on this I suggest following https://github.com/lemonade-sdk/lemonade and watch the supported configurations table for LINUX NPU support, at that point ROCm or Vulkan will probably have working AMD NPU support.

Edit: That isn't to say ROCm won't have it before then but if it is good enough for them to implement it probably is good enough and ready for others to implement.

[spattinson]

Not sure if that dragon repo is real/genuine it seems to good to be true see In2infinity/dragon-npu#5

The current frigate 0.16 don't support GFX1150 which is what HX370 is, you can override to 1100 and it works.

I installed dev version 0.17.0-a8bcc109 after changing it to have GFX1150 and noticed that rocminfo shows 3 agents whereas the rocminfo in frigate 0.16 with GFX1100 only shows two agents. IDK if one of them is the NPU (AIE-ML)
I am getting about 9ms inference with four yolov9-s-320.onnx detectors. There is a bug in reporting GPU usage at always 100%, that issue goes away if you use two detectors but inference time goes up a bit.

[chicheng]

AMD just merged NPU on Linux doc (not published yet)
Driver: https://github.com/amd/xdna-driver

[Gibbz]

I have an amd npu and would love to see this working 👍

[putt6359]

It would be interesting to see if ROCm can use it directly or not. The main problem with adding support is that we don't have any devices with NPUs built in to test with

Is this still true to this day? How much cheapest AMD NPU can we provide so this could get started?

[NickM-27]

A user has successfully run with the NPU
#21906 (comment)