Skip to content

Commit 7126784

Browse files
authored
【Fix】fix mtp bug (#3139)
1 parent b76b17f commit 7126784

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

fastdeploy/model_executor/layers/sample/sampler.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -447,8 +447,8 @@ def forward_cuda(
447447
sampling_metadata.min_dec_lens,
448448
sampling_metadata.eos_token_ids,
449449
share_inputs["seq_lens_this_time"],
450-
share_inputs["seq_lens_encoder"],
451-
share_inputs["seq_lens_decoder"],
450+
share_inputs["output_padding_offset"],
451+
share_inputs["output_cum_offsets"],
452452
max_model_len,
453453
)
454454
probs = F.softmax(logits)

0 commit comments

Comments
 (0)