Can we run MLX on Docker(with GPU acceleration/METAL) on MacOS ?

[AndriiTsok]

I am wondering if that is possible to achieve GPU acceleration on mac on docker container and run MLX inference with the acceleration?

[vade]

I don't believe so. Last I read up on this, Docker requires specific Linux kernel niceties ( cgroups ) which Darwin (macOS Kernel) does not support. This means that Docker on macOS runs a virtual machine emulating x86_64 or arm64, which runs a variant of a Linux OS using a kernel that supports cgroups

Since there are no Linux Metal drivers, no go at all.

That said, Apple has been working on their visualization framework, so in theory its cheap to launch a secondary macOS instance in a VM which would have full GPU / Metal, but, it isnt containerized or support docker files or docker-compose.

I sincerely wish Apple would adopt cgroups, and let licensing for macOS happen to support proper orchestration and containerization IAC workflows. Alas.