Skip to content

Commit 9cacd66

Browse files
committed
Fix MRoPE model inference when no MM embeddings present
1 parent 69bb9d6 commit 9cacd66

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

exllamav2/model.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -954,7 +954,7 @@ def forward_chunk(
954954
seq_len <= self.config.max_output_len, \
955955
"seq_len exceeds max_output_len"
956956

957-
if self.config.arch.lm.mrope and "indexed_embeddings" in kwargs:
957+
if self.config.arch.lm.mrope and kwargs.get("indexed_embeddings"):
958958
assert attn_params is not None and (
959959
attn_params.rope_offsets is not None or
960960
attn_params.get_alt_rope_embed("cpu") is not None

0 commit comments

Comments
 (0)