Adding IQ1_KT - 1.75 bpw SOTA quants#616

Merged

ikawrakow merged 14 commits intomainfrom

Jul 20, 2025

Commits on Jul 16, 2025

iq1_kt: basics
Iwan Kawrakow
committed
iq1_kt: CUDA dequantize
Iwan Kawrakow
committed
iq1_kt: CUDA MMQ
Iwan Kawrakow
committed
iq1_kt: CUDA MMVQ
Iwan Kawrakow
committed
iq1_kt: AVX2 GEMM/GEMV
Iwan Kawrakow
committed
iq1_kt: convert/repack to q8_0_r8 (AVX2)
Iwan Kawrakow
committed
iq1_kt: slightly faster GEMV
Iwan Kawrakow
committed
iq1_kt: NEON GEMM/GEMV
Iwan Kawrakow
committed
iq1_kt: slightly faster NEON - still pathetic
Iwan Kawrakow
committed
iq1_kt: tiny bit better GEMV on NEON
Iwan Kawrakow
committed
iq1_kt: convert/repack to q8_0_r8 (NEON)
Iwan Kawrakow
committed
iq1_kt: very slightly faster convert/repack to q8_0_r8 on NEON
Iwan Kawrakow
committed
Adding frgotten file
Iwan Kawrakow
committed

Commits on Jul 17, 2025

iq1_kt: add to constants.py
Iwan Kawrakow
committed