[WIP] Attempt to run on Linux

[Joannis]

Proposed changes

Please include a description of the problem or feature this PR is addressing. If there is a corresponding issue, include the issue #.

Checklist

Put an x in the boxes that apply.

• I have read the CONTRIBUTING document
• I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
• I have added tests that prove my fix is effective or that my feature works
• I have updated the necessary documentation (if needed)

[Joannis]

Sorry for the maintainers, I pushed this WIP code here for public tracking of a bug I'm sorting first. I'm assuming you're on the same team

[davidkoski]

Sorry for the maintainers, I pushed this WIP code here for public tracking of a bug I'm sorting first. I'm assuming you're on the same team

Yep, same people :-)

[Joannis]

@davidkoski this is ready for review. I've added CI (wasn't able to run it on my fork for some reason).

Highly recommend disabling whitespace when viewing this PR. I've added a fair amount of #if os(macOS) || os(iOS) || os(tvOS) || os(watchOS) || os(visionOS) checks around where I couldn't run it on Linux (yet). Formatting made sure that this blows up the diff.

With this change, I was able to run MLXLM, (not VLM) on Linux. I did have to use a custom build with GPU acceleration to run this, as I couldn't use it on CPU mode due to this: ml-explore/mlx#3080

The float16 issue is also present on my macOS device if I run it in docker. So unfortunately I can't make an end-to-end Docker example until we solve CUDA. Working on that though

[davidkoski]

Yeah, the VLM code probably needs some thinking to separate out the image processing and image representation. The models should all be dealing with MLXArray, hopefully we can just replace MediaProcessing.

[davidkoski]

We shouldn't need this -- MLXFast is an empty package with some stubs to forward back to MLX. MLX has an enum MLXFast so without import MLXFast the same code should still compile.

[Joannis]

Good one yeah. It's an artifact since this code has been on my machine for a while. Will rid of it

[Joannis]

This doesn't work on Linux yet