NVIDIA Detection Speed

[lockkp1]

Was just curious how Frigate object detection worked in general and wondering if switching to a GPU system would change anything for me. So currently running Frigate on an M1 Mac mini using docker. Getting inference speeds of 10ms on average with 640x480, at 5FPS, 7 cameras in total. If there is heavy motion on multiple cameras at once it can reach inference speeds of 150ms for a brief moment but mostly hovers from the 20-50ms. For anyone curious I tried upping the resolution to 720p, 5FPS, on just 1 camera while keeping the other 6 constant at 640x480, and it struggles. Inference speed goes to 60-200ms.

My issue with the current setup though is that I find object detection to be a bit slow so I wasn't sure if this was a M1 limitation with decoding FFMPEG? I'm mainly using it for presence and motion detection, so tracking people and activating lights via motion. When I watch some of the events with what I would consider very ideal conditions for the camera, (sunny environment, with a very clear object), confirmation for a person via MQTT can at times be quite slow, 3-5 seconds later and it will send an MQTT person motion message, but sometimes the person has already passed. I thought maybe it was issues with certain cameras (POE HIKVISION cameras), however the same cameras will at other times have quick detection. Its just not stable some events are fast and others are slow. I even adjusted the motion threshold to 15 and then 10, but the M1 really struggles when I do that, inference jumps to 20ms+ for steady, and then when motion is detected averages at 150-200ms.

I was wondering then if I switched to an amd64 with GPU would I see any difference? I've got a 1070 GPU lying around but no CPU yet, so I'm just curious if a GPU system equipped with a coral would detect a person quicker? Also would it breeze through higher resolutions?

[blakeblackshear]

Frigate doesn't support using a GPU for object detection at all yet, so I think everything will be worse. Interesting to know that the M1 struggles to maintain low inference speeds when CPU usage goes up from other processes. I don't see that with a Coral. Are you using a Coral on the Mac mini?

[lockkp1]

Thanks Blake. Sorry wanted clarify I was meaning using gpu on an amd64 and using it for ffmpeg decoding. I think the issue with the M1 maybe that decoding maybe slow. Then top that off with having to detect so maybe that also explains why inference goes up during heavy motion?

When I set the camera to debug and watch it the object detection seems quite fast so I think maybe it may be a problem with decoding?

In regards to the coral not using one yet. This is solely on M1 for now but if I switch to amd64-nvidia then I would purchase one to use with it. Just not sure right now if it’s worth getting a whole new system haha

[blakeblackshear]

Frigate should be sending mqtt messages instantly as you watch the debug view. Do you see a significant delay there?

[lockkp1]

No delay during debug mode. The mqtt message delays are really obvious when camera is just normally operating. So basically either when I watch it after the fact or when using it as a motion detector.

I’ve also got it setup as a motion detector for my garage and that sometimes one works instantly and other times delayed.

[blakeblackshear]

You are watching the mqtt messages with something like MQTT explorer and they are delayed? Note that frigate does not send any messages until it is a confirmed true positive, which takes at least 3 frames.