Faster MLA prompt processing#205

Merged

ikawrakow merged 8 commits intomainfrom

Feb 13, 2025

Commits on Feb 12, 2025

Do not allocate / report caches that are not used
Iwan Kawrakow
committed
Rename X_pe to X_rope
Iwan Kawrakow
committed
WIP
Iwan Kawrakow
committed
WIP
Iwan Kawrakow
committed
WIP
Iwan Kawrakow
committed
WIP
Iwan Kawrakow
committed

Commits on Feb 13, 2025

Warn user when disabling MLA
Iwan Kawrakow
committed
MLA: compile time option to not use transposed KV cache
Iwan Kawrakow
committed