Add optional MLA#188

Merged

ikawrakow merged 6 commits intomainfrom

Feb 9, 2025

Commits on Feb 8, 2025

Deepseek MLA Optimizations

authored and
Iwan Kawrakow
committed
Make MLA optional
Iwan Kawrakow
committed
Remove some unnecessary copies in the MLA attention
Iwan Kawrakow
committed

Commits on Feb 9, 2025

Deepseek MLA Optimizations V2 (#195)

authored
Use type_k and type_v to set the types of the MLA caches
Iwan Kawrakow
committed
Better gemm strategy when nth > nhead
Iwan Kawrakow
committed