Simple load balancing for immich_machine_learning using nginx, with support for mac (CoreMLExecutionProvider, ANE)

[apetersson]

Ever wondered, can i speed up machine learning using some spare hardware such as Macbooks, Desktops etc.?
The built-in machine learning config appears to only work in a round-robin fashion. It's nice to find "fallbacks" but it's not designed for leveraging all your hardware at once. The setup to do this is not that complicated, but there are some details to take care of.

"just one more container, mom, then our Immich setup will be perfect" - by extending your docker-compose you can instantiate a immich_ml_balancer which is easy to configure a list of potential immich_machine_learning instances and with the simple logic nginx "least_conn" distribute load to all of them. check out

https://github.com/apetersson/immich_ml_balancer

for a complete how to, including getting a native ANE-enabled setup to run.