DeepSeek CUDA Flash Attention #241

Merged

ikawrakow merged 6 commits intomainfrom

ik/cuda_fattn_Dk_Dv

Mar 5, 2025

Commits on Mar 3, 2025

WIP CUDA FA with Dk != Dv
Iwan Kawrakow
committed
WIP
Iwan Kawrakow
committed
CUDA FA WIP - It actually works!
Iwan Kawrakow
committed
CUDA FA WIP - it now works for Q8_0 + Q8_0 for KV cache
Iwan Kawrakow
committed
CUDA FA WIP - TG, not working yet.
Iwan Kawrakow
committed

Commits on Mar 4, 2025

CUDA FA with Dk != Dv: it works now for DeepSeek
Iwan Kawrakow
committed