Commit 7ac15b0
committed
Update on "[Executorch][llm] Add ring buffer based kv cache and mask calculation to MHA"
Leveraging previous work now we allow MHA to have ring buffer cache. If ring buffer cache is used
then we query the mask from kv cache and use that for sdpa instead of using precalculated mask.
In this process we had to adjsut ring buffer implementation to allow keeping the context of
full sliding window. See code for comment.
Differential Revision: [D73891425](https://our.internmc.facebook.com/intern/diff/D73891425/)
[ghstack-poisoned]File tree
0 file changed
+0
-0
lines changed0 file changed
+0
-0
lines changed
0 commit comments