Commit 4b18555
Fix text_llm_runner kv cache pos count and use it for generate() (#15295)
### Summary
pos_ should advance by prefill and generated prompt size.
### Test plan
CI
cc @larryliu0820 @mergennachin @cccclai @helunwencser @jackzhxng
Co-authored-by: Hansong Zhang <[email protected]>1 parent 8c84780 commit 4b18555
1 file changed
+3
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
183 | 183 | | |
184 | 184 | | |
185 | 185 | | |
186 | | - | |
| 186 | + | |
187 | 187 | | |
188 | 188 | | |
189 | 189 | | |
190 | 190 | | |
| 191 | + | |
| 192 | + | |
191 | 193 | | |
192 | 194 | | |
193 | 195 | | |
| |||
0 commit comments