Trying to load Mamba GGUF gives GGML_ASSERT(ggml_is_matrix(c)) Failed error

[Macktek]

I have tried several Mamba based GGUF, and am getting this error: ggml/src/ggml.c:4461: GGML_ASSERT(ggml_is_matrix(c)) Failed error

Is Koboldcpp compatible with Mamba based GGUF? (If so, can you suggest one).

[LostRuins]

The last time I checked it was. Which model did you try

[Macktek]

Thanks for the reply. See the failure models below:

Failed today: mradermacher/Falcon3-Mamba-R1-v0-GGUF

And these yesterday:
PrunaAI/state-spaces-mamba-2.8b-hf-GGUF-smashed
mradermacher/mamba-2.8b-papers-trained-GGUF
bartowski/mamba-2.8b-hf-GGUF

This one did load: ( Triangle104/Falcon3-Mamba-7B-Instruct-Q8_0-GGUF (tho it rambles after a bit)

[LostRuins]

I just tried mamba-2.8b-hf-Q2_K and it worked fine. Though it was slow. Which quant did you use?

[Macktek]

mamba-2.8b-hf.Q5_K_S.gguf
I don't even see the option for a Q2_K: https://huggingface.co/bartowski/mamba-2.8b-hf-GGUF/tree/main
Smallest I can see is mamba-2.8b-hf-IQ3_M.gguf
( I will try to hunt down that mamba-2.8b-hf-Q2_K tho)

[Macktek]

( I did find a smaller one: leliuga/mamba-2.8b-hf-GGUF (which does work, but again it repeats itself and rambles after a few paragraphs).
Any luck getting any of the broken ones that I listed to work?

[LostRuins]

Here is a direct link to the file: https://huggingface.co/bartowski/mamba-2.8b-hf-GGUF/resolve/main/mamba-2.8b-hf-Q2_K.gguf

trying the q5_k_s works too. So there is nothing wrong with that file.

[Macktek]

I can confirm that Bartowski Mamba model loads.
But did you try any of the ones that are giving me that error:

Look at this Q5 one: https://huggingface.co/PrunaAI/state-spaces-mamba-2.8b-hf-GGUF-smashed/tree/main/state-spaces
Direct Link: https://huggingface.co/PrunaAI/state-spaces-mamba-2.8b-hf-GGUF-smashed/resolve/main/state-spaces/mamba-2.8b-hf.Q5_K_S.gguf?download=true