Skip to content

Commit 1e01498

Browse files
authored
fix bug for pd mode, decode node need set finish status to time out reqs. (#833)
1 parent 757644c commit 1e01498

File tree

1 file changed

+1
-0
lines changed
  • lightllm/server/router/model_infer/mode_backend/continues_batch/pd_mode/decode_node_impl

1 file changed

+1
-0
lines changed

lightllm/server/router/model_infer/mode_backend/continues_batch/pd_mode/decode_node_impl/decode_impl.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,7 @@ def _post_init_reqs(self, uninit_reqs: List[InferReq]):
114114
req_obj.init_all()
115115
req_obj.set_next_gen_token_id(0, 0.0)
116116
req_obj.cur_output_len += 1
117+
req_obj.finish_status.set_status(FinishStatus.FINISHED_STOP)
117118

118119
if self.is_master_in_dp:
119120
req_obj.shm_req.shm_cur_kv_len = req_obj.cur_kv_len

0 commit comments

Comments
 (0)