Faster CPU prompt processing for Q4_K and Q5_K#525

Merged

ikawrakow merged 4 commits intomainfrom

Jun 13, 2025

Commits on Jun 12, 2025

q4_K: dequantize to q8_1_r8 for batch >= 32
Iwan Kawrakow
committed
q4_K: GEMM with q8_2_X4
Iwan Kawrakow
committed
q5_K: GEMM with q8_2_X4 and repack to q8_1_r8
Iwan Kawrakow
committed
Remove the scales, they are not needed
Iwan Kawrakow
committed