mlx: Add initial mlx implementation

[tmc]

This adds an initial adaptation of the jax and pytorch implementations to mlx.

[artus-LYTiQ]

Lol, you beat me to it. Will check and compare

[artus-LYTiQ]

gemini_kvcache.py.txt
gemini_main.py.txt
gemini_model.py.txt
gemini_sampler.py.txt
gemini_stats.py.txt
gemini_weights.py.txt

Hope you can make something out of the attached files. I tried to one-shot the Jax code to MLX as it is really quite similar. But the subtle differences of course let that fail. Then I fiddled around with Sonnet 3.5 until it ran but it is obviously buggy and my brain is fried for today. @tmc

[samefarrar]

If you're interested, and xjdr would rather keep them seperate, I have a MLX fork that runs on my M2 Macbook out the box. Have implemented server, SSE, and a frontend for changing system prompts etc. still WIP though but caught up to the main branch (frog branch incoming).

[artus-LYTiQ]

Stuck at work for the next five or six hours but would love to integrate the three efforts (or push the most promising further). The core question will imo be how xjdr will want to treat imperfect/deviating implementations and how fast we can come up with tooling that measures how big those deviations actually are.

[d-r-w]

If you're interested, and xjdr would rather keep them seperate, I have a MLX fork that runs on my M2 Macbook out the box. Have implemented server, SSE, and a frontend for changing system prompts etc. still WIP though but caught up to the main branch (frog branch incoming).

Your fork is really cool but as you mention it's very different
Love the idea of a out-of-box adapter assuming there aren't random bugs introduced by claude