Skip to content

Commit 201c971

Browse files
authored
[Perf][Easy] Early stop in request_block_hasher (#26112)
Signed-off-by: Jialin Ouyang <[email protected]>
1 parent e0986ea commit 201c971

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

vllm/v1/core/kv_cache_utils.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -585,6 +585,10 @@ def request_block_hasher(request: Request) -> list[BlockHash]:
585585
start_token_idx = len(request.block_hashes) * block_size
586586
num_tokens = request.num_tokens
587587

588+
if start_token_idx + block_size > num_tokens:
589+
# Early stop when there no new full blocks created.
590+
return []
591+
588592
curr_mm_idx = 0
589593
if start_token_idx > 0:
590594
# Set curr_mm_idx = -1 to indicate the last mm input.

0 commit comments

Comments
 (0)