Add support for IQn_K quants

[arnfaldur]

@ikawrakow has created new quantization schemes IQ2_K, IQ3_K, IQ4_K, IQ5_K (ikawrakow/ik_llama.cpp#8) and IQ6_K (ikawrakow/ik_llama.cpp#14) that together constitute a new pareto frontier, based on his perplexity tests.

Interestingly, he found that Q6_K caused considerable quality loss, contrary to popular belief, when quantizing newer models. Theorizing that they contain more information in their weights and are therefore more sensitive to quantizations. IQ6_K reduced that quality loss from 0.65% to 0.4%.

High quality qantizations, combined with relative ease of use, are the main differentiator llama.cpp has with it's competitors from my, and likely many others', perspective.

Supporting these new quantization schemes would be a great addition to the project and would help llama.cpp stay in it's class of it's own level of quality.

@ikawrakow already implemented all the quants in his fork ikawrakow/ik_llama.cpp#7 but it has diverged significantly.

There seem to have been some disagreements that lead to this fork being made. But I think everyone can understand the value in upstreaming this work, and finding a path that would lead to that happening.

[Orion-zhen]

Yes, and there are lots of things that ik_llama.cpp is better than llama.cpp. I believe it would be of great benefit if someone could port ik_llama.cpp's feature back into llama.cpp. I have tried latest IQn_KT quants, and they really impressed me a lot.

[abc-nix]

Every time someone tries to bring a feature from ik_llama to llama.cpp, the fork owner (ikawrakow) makes a fuss about attributions. I will not go into the history, but the passive-aggressive behavior of both mainline and the fork has made it so that the llama.cpp repository owners will not even look at the code of ik_llama, in fear of being accused of plagiarizing (at least that is my interpretation).