fix: fix a bug in flashinfer_struct

niushengxiao · niushengxiao · commit b3592524c11d · 2025-07-09T20:14:44.000+08:00
diff --git a/lightllm/models/llama/flashinfer_struct.py b/lightllm/models/llama/flashinfer_struct.py
@@ -81,7 +81,7 @@ def init_some_extra_state(self, model, input_ids: torch.Tensor):
                     self.req_manager.req_to_token_indexs,
                     self.b_req_idx,
                     self.b_seq_len,
-                    self.b_start_loc,
+                    kv_starts,
                     self.max_len_in_batch,
                     kv_indices,
                 )

Original file line number	Diff line number	Diff line change
`@@ -81,7 +81,7 @@ def init_some_extra_state(self, model, input_ids: torch.Tensor):`
`81`	`81`	`self.req_manager.req_to_token_indexs,`
`82`	`82`	`self.b_req_idx,`
`83`	`83`	`self.b_seq_len,`
`84`		`- self.b_start_loc,`
	`84`	`+ kv_starts,`
`85`	`85`	`self.max_len_in_batch,`
`86`	`86`	`kv_indices,`
`87`	`87`	`)`