How to use mlx.core.save_gguf? / How to save as .gguf file?

[mark-lord]

Hi all! Unfortunately I'm struggling again a bit with the docs; I can't find a detailed enough explanation for how to use the mlx.core.save_gguf command so I'm confused as to how to convert to a .gguf. I did try GPT-4 but alas, it was of no help. Would really love to be able to convert to a 4 or even 8 bit .gguf file so I hope this is possible :) - if it's with a different command please let me know!

Thanks everyone 🙌

[awni]

You would use it like this:

        save_dict = {"test": mx.ones((4, 4), dtype=mx.int32)}
        mx.save_gguf("array.gguf", save_dict)

[mark-lord]

Awesome, thank you 😊 Will implement this