请问一下在prefill计算时，ProcessKV和FlattenKV的作用分别是什么？ #2063
Unanswered
sitabulaixizawaluduo
asked this question in
Q&A
Replies: 1 comment 3 replies
-
|
是的 |
Beta Was this translation helpful? Give feedback.
3 replies
-
|
我看到ProcessKV和FlattenKV分别做了一次rope，FlattenKV中把kv cache从block中取出来时为什么还要再做一次rope呢，这个操作是针对什么场景的？ |
Beta Was this translation helpful? Give feedback.
-
|
同样好奇 |
Beta Was this translation helpful? Give feedback.
-
|
这两个地方不会同时生效 |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
是不是可以理解ProcessKV是把经过qkv_proj计算的k v存到block里面，FlattenKV是把block kv cache转为整块的kv cache参与后续的attention计算？
Beta Was this translation helpful? Give feedback.
All reactions