[DRAFT] Use custom SDPA in Llama 3.2 MM#7471

Closed

jackzhxng wants to merge 13 commits intomainfrom

Commits on Dec 11, 2024

Add tests that localize the prefill issue to the kv cache
jackzhxng
committed

Commits on Dec 13, 2024

Fixes test but not model
jackzhxng
committed

Commits on Dec 16, 2024

Updated pass
jackzhxng
committed

Commits on Dec 18, 2024

Commits on Dec 21, 2024

PR review
jackzhxng
committed
Fix test
jackzhxng
committed

Commits on Dec 23, 2024

Last changes
jackzhxng
committed

Commits on Jan 2, 2025

Custom SDPA in attention
jackzhxng
committed