Skip to content

Commit 0b225fb

Browse files
authored
[Misc] skip target model mm emb in draft proposal step when draft is text-only (vllm-project#33437)
Signed-off-by: kkt-cohere <[email protected]>
1 parent 46b4a02 commit 0b225fb

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/v1/worker/gpu_model_runner.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4070,7 +4070,7 @@ def propose_draft_token_ids(
40704070
else:
40714071
target_hidden_states = hidden_states[:total_num_tokens]
40724072

4073-
if self.supports_mm_inputs:
4073+
if self.supports_mm_inputs and self.drafter.supports_mm_inputs:
40744074
mm_embed_inputs = self._gather_mm_embeddings(
40754075
scheduler_output,
40764076
shift_computed_tokens=1,

0 commit comments

Comments
 (0)