Example for how to load GGUF inside a Python program?

[atipasvanund]

Is there something similar to this?

from mlx_lm import load

model, tokenizer = load("mode.gguf")

The examples I can find shows how to do it from console but not how to do it from within a Python program.

[awni]

You can use mx.load to load any gguf file. It will give you two dictionaries, one for the metadata and one for the arrays themselves.

This function in MLX Examples shows how to use the GGUF to make an LLM + tokenizer.