Skip to content

Commit 66fa4e2

Browse files
author
wangzaijun
committed
fix
1 parent 0c605ef commit 66fa4e2

File tree

1 file changed

+1
-1
lines changed
  • lightllm/server/router/model_infer/mode_backend/pd_nixl/decode_node_impl

1 file changed

+1
-1
lines changed

lightllm/server/router/model_infer/mode_backend/pd_nixl/decode_node_impl/decode_impl.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,7 @@ def _decode_node_gen_trans_tasks(self, req_obj: InferReq):
128128
"""
129129
decode node 生成所有的传输任务对象。
130130
"""
131-
# 传输的 kv 要少一个,不然decode 无法有下一个输入除非推理出下一个token
131+
# 传输的 kv 要少一个,不然decode 无法有下一个输入推理出下一个token
132132
input_len = req_obj.shm_req.input_len - 1
133133
page_size = self.args.nixl_pd_kv_page_size
134134
req_obj.nixl_trans_kv_start_index = req_obj.cur_kv_len

0 commit comments

Comments
 (0)