Do you plan to implement Slim Attention?

[albertosottile]

Or would you wait for upstream implementation in llama.cpp?

Context: https://www.reddit.com/r/LocalLLaMA/comments/1j9wkc2/slim_attention_cut_your_context_memory_in_half

Issue in llama.cpp: ggml-org#12359

Implementation in Python: https://colab.research.google.com/github/OpenMachine-ai/transformer-tricks/blob/main/notebooks/slimAttn_paper.ipynb

[LostRuins]

Definitely upstream implementation is preferred