New IQ2_KT#511

Closed

ikawrakow wants to merge 26 commits intomainfrom

Commits on Jun 7, 2025

New iq4_kt trellis
Iwan Kawrakow
committed
Something is not working with the AVX2 dot product
Iwan Kawrakow
committed
New iq4_kt: CUDA MMVQ
Iwan Kawrakow
committed
New iq4_kt: CUDA MMQ
Iwan Kawrakow
committed
For now have only iq4_kt use the new trellis
Iwan Kawrakow
committed
Fix iq2_kt that got broken along the way
Iwan Kawrakow
committed
New iq4_kt: AVX2 dot product finally works
Iwan Kawrakow
committed
New iq4_kt: fix vanilla AVX2
Iwan Kawrakow
committed

Commits on Jun 8, 2025

New iq4_kt: NEON implementation
Iwan Kawrakow
committed
New iq4_kt: slightly faster NEON
Iwan Kawrakow
committed
New iq4_kt: slightly faster NEON
Iwan Kawrakow
committed
New iq4_kt: faster NEON
Iwan Kawrakow
committed
Minor
Iwan Kawrakow
committed
New iq4_kt trellis: not working Metal implementation
Iwan Kawrakow
committed
Remove the extra 4 bytes of row meta data that is no longer used
Iwan Kawrakow
committed
Cleanup
Iwan Kawrakow
committed

Commits on Jun 9, 2025

Adding forgottent file
Iwan Kawrakow
committed
Switching iq2_kt to new trellis - CUDA MMQ
Iwan Kawrakow
committed
New iq2_kt: CUDA GEMV
Iwan Kawrakow
committed
New iq2_kt: AVX2 dequantize
Iwan Kawrakow
committed
New iq2_kt: AVX2 GEMM/GEMV
Iwan Kawrakow
committed
Adding forgotten file
Iwan Kawrakow
committed
New iq2_kt: NEON GEMM/GEMV
Iwan Kawrakow
committed
New iq2_kt: slightly faster NEON GEMM
Iwan Kawrakow
committed
New iq2_kt: Metal - very slow.
Iwan Kawrakow
committed
Add missing break
Iwan Kawrakow
committed