llama : support converting Mistral Small (text-only)

[ngxson]

Important

At the time of writing this, HF model is missing chat template, let's wait until this PR is merged!

This PR allows converting text model from Mistral Small to GGUF

For vision, I don't know yet how it work. Will work it that later.

And congrats to @mistralai team for the release

[bartowski1182]

chat template is merged, LGTM!

[ngxson]

@bartowski1182 Waiting for your gguf quant to test hehe

[bartowski1182]

merging locally to check :)

[bartowski1182]

quant worked: https://huggingface.co/lmstudio-community/Mistral-Small-3.1-24B-Instruct-2503-GGUF

verifying locally that it runs without issue (though imatrix is working which is usually a good sign)

[bartowski1182]

looks like it worked! but something went wrong with chat template so gonna have to redo that part :')

[bartowski1182]

oh whoops, i was looking at the wrong chat template PR merge!

[bartowski1182]

Okay, merged your changes and the chat template locally (minus the "today's date" portion which doesn't play nice usually), and uploaded the quants:

https://huggingface.co/lmstudio-community/Mistral-Small-3.1-24B-Instruct-2503-GGUF

They run in LM Studio so i think we're safe to say it works !

[ngxson]

Nice, thanks for testing. Now I'm waiting for approval to merge this.

[Kaszebe]

Slowpoke here...image recognition won't work in LM Studio until ggerganov merges the changes?

[bartowski1182]

@Kaszebe it won't work until @ngxson works some (powerful) magic and figures it out, it's non-trivial to add.

Thankfully text version of this seems straightforward so we get to play around with it until vision is figured out later :) (assume days minimum, possibly weeks depending on difficulty)