Skip to content

Commit 7ebb46b

Browse files
committed
Fix text_llm_runner kv cache pos count
1 parent 58bd91c commit 7ebb46b

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

extension/llm/runner/text_llm_runner.cpp

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,8 @@ Error TextLLMRunner::generate(
199199
}
200200
int64_t num_generated_tokens = generate_result.get();
201201

202+
pos_ += num_generated_tokens;
203+
202204
stats_->inference_end_ms = time_in_ms();
203205
if (!config.warming) {
204206
printf("\n");

0 commit comments

Comments
 (0)