Commit 6b4b933
committed
Update base for Update on "[Executorch][llm] Add ring buffer based kv cache and mask calculation to MHA"
Leveraging previous work now we allow MHA to have ring buffer cache. If ring buffer cache is used
then we query the mask from kv cache and use that for sdpa instead of using precalculated mask.
In this process we had to adjsut ring buffer implementation to allow keeping the context of
full sliding window. See code for comment.
Differential Revision: [D73891425](https://our.internmc.facebook.com/intern/diff/D73891425/)
[ghstack-poisoned]1 parent 2543fe7 commit 6b4b933
File tree
0 file changed
+0
-0
lines changed0 file changed
+0
-0
lines changed
0 commit comments