Skip to content

Commit c24be03

Browse files
[bugfix] kvcomp for qwen (#594)
1 parent 6899198 commit c24be03

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

ucm/sparse/kvcomp/kvcomp_hbm.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -250,8 +250,9 @@ def attention_begin(
250250
k_hash_compute = self.hash_encoder.compute_hash(key).view(
251251
torch.bfloat16
252252
)
253+
valid_k_hash_token = attn_metadata.slot_mapping.flatten().numel()
253254
reshape_and_cache_khash_triton(
254-
k_hash_compute,
255+
k_hash_compute[:valid_k_hash_token],
255256
attn_metadata.slot_mapping.flatten(),
256257
k_hash,
257258
block_size=self.block_size,

0 commit comments

Comments
 (0)