Skip to content

Commit d783c26

Browse files
kylesayrsdsikka
andauthored
[VLM] Fix mllama targets (#1402)
## Purpose ## * When #1389 landed, modules being skipped by ignore were no longer being skipped. However, this requires that the sequential targets list be correct. Mllama defaults to targeting vision layers, and hence the vision tower was being traced, leading to errors. ```python3 _no_split_modules = [ "MllamaVisionEncoderLayer", "MllamaCrossAttentionDecoderLayer", "MllamaSelfAttentionDecoderLayer", ] ``` ## Changes ## * Only target text decoder layers, not vision decoder layers ## Testing ## * #1335 passes Signed-off-by: Kyle Sayers <kylesayrs@gmail.com> Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
1 parent 564140d commit d783c26

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

examples/multimodal_vision/mllama_example.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ def data_collator(batch):
3232
GPTQModifier(
3333
targets="Linear",
3434
scheme="W4A16",
35+
sequential_targets=["MllamaSelfAttentionDecoderLayer"],
3536
ignore=["re:.*lm_head", "re:multi_modal_projector.*", "re:vision_model.*"],
3637
),
3738
]

0 commit comments

Comments
 (0)