Bitnet changes#106

Merged

ikawrakow merged 12 commits intomainfrom

ik/adapt_iq1_iq2_bn

Oct 25, 2024

Commits on Oct 24, 2024

Adapting iq2_bn to work without separate scale tensors
Iwan Kawrakow
committed
Adapting iq1_bn to work without separate scale tensors
Iwan Kawrakow
committed
Adapting iq2_bn: CUDA dequantize
Iwan Kawrakow
committed
Adapting iq2_bn: CUDA works
Iwan Kawrakow
committed
Adapting iq1_bn: CUDA works
Iwan Kawrakow
committed
Adapting iq1_bn, iq2_bn: NEON
Iwan Kawrakow
committed
Adapting iq1_bn, iq2_bn: Metal
Iwan Kawrakow
committed
WIP
Iwan Kawrakow
committed
Remove iq1_tn and iq2_tn - Part 1
Iwan Kawrakow
committed
Remove iq1_tn and iq2_tn - Part 2
Iwan Kawrakow
committed
Bitnet: use the standard llm_build_kv to build self attention
Iwan Kawrakow
committed

Commits on Oct 25, 2024

Revert "Avoid rebuild of GGML graph for each token (#98)"
Iwan Kawrakow
committed