Skip to content

Commit 90e72ae

Browse files
DongheJinRobbieLeung
authored andcommitted
bugfix: correct kv_cache shape in DeepSeek. (#560)
1 parent 7010751 commit 90e72ae

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

xllm/core/runtime/llm_engine.cpp

100755100644
Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -263,7 +263,8 @@ Engine::KVCacheCapacity LLMEngine::estimate_kv_cache_capacity() {
263263
slot_size =
264264
dtype_size *
265265
((args_.kv_lora_rank() + NZ_ALIGNMENT - 1) / NZ_ALIGNMENT +
266-
(args_.qk_rope_head_dim() + NZ_ALIGNMENT - 1) / NZ_ALIGNMENT);
266+
(args_.qk_rope_head_dim() + NZ_ALIGNMENT - 1) / NZ_ALIGNMENT) *
267+
NZ_ALIGNMENT;
267268
} else {
268269
slot_size =
269270
dtype_size * (args_.kv_lora_rank() + args_.qk_rope_head_dim());

0 commit comments

Comments
 (0)