Evaluate falcon3 family

[the-crypt-keeper]

https://huggingface.co/collections/tiiuae/falcon3-67605ae03578be86e4e87026

The falcon has landed

[the-crypt-keeper]

Complete. A bit of an odd result: Every GGUF except 7B is behind the FP16, while the 7B GGUF mysteriously outperforms even 10B FP16.

[the-crypt-keeper]

ggml-org/llama.cpp#10876

It's been reverted due to problems with the tokenizer, going to have to re-run GGUFs once it's fixed

[the-crypt-keeper]

Done on latest GGUF with latest llama.cpp, added ollama as well.