New IQ4_KT trellis implementation#505

Closed

ikawrakow wants to merge 17 commits intomainfrom

Commits on Jun 7, 2025

New iq4_kt trellis
Iwan Kawrakow
committed
Something is not working with the AVX2 dot product
Iwan Kawrakow
committed
New iq4_kt: CUDA MMVQ
Iwan Kawrakow
committed
New iq4_kt: CUDA MMQ
Iwan Kawrakow
committed
For now have only iq4_kt use the new trellis
Iwan Kawrakow
committed
Fix iq2_kt that got broken along the way
Iwan Kawrakow
committed
New iq4_kt: AVX2 dot product finally works
Iwan Kawrakow
committed
New iq4_kt: fix vanilla AVX2
Iwan Kawrakow
committed

Commits on Jun 8, 2025

New iq4_kt: NEON implementation
Iwan Kawrakow
committed
New iq4_kt: slightly faster NEON
Iwan Kawrakow
committed
New iq4_kt: slightly faster NEON
Iwan Kawrakow
committed
New iq4_kt: faster NEON
Iwan Kawrakow
committed
Minor
Iwan Kawrakow
committed
New iq4_kt trellis: not working Metal implementation
Iwan Kawrakow
committed
Remove the extra 4 bytes of row meta data that is no longer used
Iwan Kawrakow
committed
Cleanup
Iwan Kawrakow
committed

Commits on Jun 9, 2025

Adding forgottent file
Iwan Kawrakow
committed