InsightFace external model

[EngTurtle]

Hi I vibe coded a fork of the external-model container to use the InsightFace models, since they were used as the basis of Immich's face clustering feature. And following immich's ML container example I added versions for Nvidia and Intel GPU compute.

https://github.com/EngTurtle/facerecognition-external-model/tree/insight-face

My quick tests of the inference speed on CPU seems similar to the original dlib models.
And the openvino testing on my N100 iGPU server was much slower than the CPU (1 min / img vs ~15s). I haven't had the chance to try it with a Nvidia GPU yet.

I'm requesting and wondering if someone can benchmark the actual detection and clustering performance, as per the original comparison https://github.com/matiasdelellis/facerecognition/wiki/Comparative-analysis-of-the-models.

[EngTurtle]

Edit: turns out openvino is much more performant for subsequent inference requests, I was restarting the container between tests and not allowing ONNX to cache the compiled openvino IR model. OpenVINO inference is 1-2s for a 2000x1500 image compared to 10-20s on CPU for the Intel N100 😁

Available providers: ['OpenVINOExecutionProvider', 'CPUExecutionProvider']
Available OpenVINO devices: ['CPU', 'GPU']
Using OpenVINO with Intel GPU GPU: {'device_type': 'GPU', 'precision': 'FP16'}
Using device: openvino
Execution providers: [('OpenVINOExecutionProvider', {'device_type': 'GPU', 'precision': 'FP16'}), 'CPUExecutionProvider']
Applied providers: ['OpenVINOExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'OpenVINOExecutionProvider': {}}
model ignore: ./models/models/buffalo_l/1k3d68.onnx landmark_3d_68
Applied providers: ['OpenVINOExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'OpenVINOExecutionProvider': {}}
model ignore: ./models/models/buffalo_l/2d106det.onnx landmark_2d_106
Applied providers: ['OpenVINOExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'OpenVINOExecutionProvider': {}}
find model: ./models/models/buffalo_l/det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['OpenVINOExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'OpenVINOExecutionProvider': {}}
model ignore: ./models/models/buffalo_l/genderage.onnx genderage
Applied providers: ['OpenVINOExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'OpenVINOExecutionProvider': {}}
find model: ./models/models/buffalo_l/w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (2048, 1536)
InsightFace model loaded successfully
[2025-10-12 21:37:55 +0000] [15] [INFO] Models preloaded successfully in worker 15
::ffff:127.0.0.1 - - [12/Oct/2025:21:38:03 +0000] 'GET /health HTTP/1.1' 200 2 in 2ms
::ffff:127.0.0.1 - - [12/Oct/2025:21:38:36 +0000] 'POST /detect HTTP/1.1' 200 36404 in 23021ms  <-- model compiling to OpenVINO IR
::ffff:127.0.0.1 - - [12/Oct/2025:21:38:36 +0000] 'GET /health HTTP/1.1' 200 2 in 0ms
::ffff:127.0.0.1 - - [12/Oct/2025:21:38:39 +0000] 'POST /detect HTTP/1.1' 200 71827 in 2448ms   <-- inference with compiled model
::ffff:127.0.0.1 - - [12/Oct/2025:21:38:41 +0000] 'POST /detect HTTP/1.1' 200 58247 in 1867ms
::ffff:127.0.0.1 - - [12/Oct/2025:21:38:42 +0000] 'POST /detect HTTP/1.1' 200 21727 in 1011ms

[EngTurtle]

moving to a more sensible 800x800 model detection size it now only takes 250ms to 1s to process an image.