Falcon-40b ggml support?

[lilblam]

How hard would it be to include support for this fork of llama that allows running falcon ggml models?

https://github.com/cmp-nct/ggllm.cpp

It's neat cuz it can run all these fine tunes:
https://huggingface.co/models?sort=modified&search=falcon+ggml

I didn't see any discussion open on it so I figure I'd ask, sorry if this was asked a billion times before somewhere!

[LostRuins]

Seems like they did make a bunch of changes to the ggml library itself. I'd need to find some time to determine how compatible these changes are

[lilblam]

ok sweet, no worries if it's too much effort. Llama 2 just released with commercial license and improved metrics, and I think the finetunes for that may dethrone Falcon in the next few weeks/months. So Falcon isn't the only capable commercial local model around anymore.

[gustrd]

Saddly Llama 2 has some limitations, like the licensing allowing use only at English. Let's see how it will go on.

[lilblam]

That's an odd licensing limitation, although it has very little training in non-English languages, so I can't imagine it would do well in anything except English anyway.

[7erminalVelociraptor]

Now all we need is ggml support for llama v2 70b.