fix: scope AWQ mapping resolution to sequential targets for multimodal models

[Yatimai]

Reported in #2102, #1577, #1415.

Root Cause

_set_resolved_mappings (line 348) deliberately does not pass the ignore list to match_modules_set, so vision encoder modules participate in matching and break the parent-context grouping.

Fix

When sequential_targets is provided (e.g. ["Gemma3DecoderLayer"]), build a _ScopedModuleView that filters named_modules() to only yield modules under those targets. This restricts match_modules_set to the text decoder, preventing vision encoder modules from polluting the LCA grouping.

• No-op when sequential_targets is not set (zero regression for text-only models)
• Respects the original author's intent: ignored modules can still be smooth layers

Testing

• Tested on Gemma 3 4B-it with AWQ W4A16 on H100
• Full calibration passed (35/35 subgraphs)
• Note: save_compressed=True fails on multimodal models due to a separate issue in compressed-tensors (non-quantized vision tower weights). The example uses save_compressed=False as a workaround.

Changes

• src/llmcompressor/modifiers/awq/base.py: scope mapping resolution via _ScopedModuleView
• examples/multimodal_vision/gemma3_awq_example.py: new AWQ example for Gemma 3

Closes #2102

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves issues related to AWQ quantization in multimodal models by ensuring that the mapping resolution process correctly targets only the relevant sequential layers, such as the text decoder. It introduces a new scoping mechanism to prevent non-textual components, like vision encoders, from being incorrectly included in the quantization mapping, which previously led to errors and tracing failures. This change improves the robustness and applicability of AWQ to complex model architectures.

Highlights

• Scoped AWQ Mapping Resolution: Introduced a mechanism to scope AWQ mapping resolution to specific sequential targets, preventing non-target modules (like vision encoders in multimodal models) from interfering with quantization mapping.
• _ScopedModuleView Implementation: Implemented a new _ScopedModuleView class that filters the modules exposed by named_modules() to only include those under specified scope prefixes, ensuring correct parent-context grouping.
• Gemma 3 Multimodal AWQ Example: Added a new example script for applying AWQ quantization to the Gemma 3 4B-it multimodal model, demonstrating the use of sequential_targets and explicit module ignores.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• examples/multimodal_vision/gemma3_awq_example.py
  • Added a new example script for applying AWQ quantization to the Gemma 3 4B-it multimodal model.
  • Demonstrated the use of sequential_targets to scope quantization to the text decoder layer.
  • Included explicit ignores for vision tower and multi-modal projector modules.
• src/llmcompressor/modifiers/awq/base.py
  • Modified _set_resolved_mappings to accept and utilize sequential_targets for scoping module resolution.
  • Introduced _ScopedModuleView and _build_scoped_model to create a filtered view of the model's modules based on specified target classes.
  • Updated on_initialize to pass sequential_targets from oneshot() kwargs or modifier fields to the mapping resolution.

Activity

• The pull request addresses multiple reported issues ([Bug]: AWQ for Gemma 3 fails to quantize, or fails to produce a viable model #2102, Gemma 3 AWQ Quantization doesnt work cache_position is NoneType #1577, [AWQ] Gemma 3: ValueError: too many values to unpack (expected 4) #1415) related to AWQ quantization in multimodal models.
• The author, gillesturpin, has provided a detailed root cause analysis and explanation of the fix.
• Testing was performed on Gemma 3 4B-it with AWQ W4A16 on H100, with full calibration passing.
• A known issue with save_compressed=True for multimodal models is noted, with a workaround provided in the example.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a well-designed fix for resolving AWQ mappings in multimodal models by scoping the module search to sequential targets. The use of _ScopedModuleView is an elegant, non-intrusive solution. The changes are logical, and the new example for Gemma 3 is a valuable addition for testing and demonstration. I have one minor suggestion to improve a comment for future maintainability.

[gemini-code-assist]

The comment mentions auto-detection from the model config, but the code below only considers kwargs and the modifier's sequential_targets field. This could be misleading for future developers. To improve clarity and maintainability, the comment should accurately describe the code's behavior.

Suggested change

	# modifier field, then auto-detect from model config.
	# modifier field.