[AWQ] Allow for activation quantization

[brian-dellabetta]

SUMMARY:
We currently error out if AWQModifier is instantiated with a scheme that includes activation quantization. This loosens that, instead issuing a warning.

Resolves #1657

TEST PLAN:
Initial tests seem promising:

"meta-llama/Llama-3.2-3B-Instruct" Wikitext PPL

W4AFP8 AWQ+Quant --  13.0959
W4AFP8 Quant -- 13.2527
W4A16 AWQ+Quant --  13.1008
W4A16 Quant -- 13.2668

Keeping track of results in

• W4fp8 AWQ #1657

[gemini-code-assist]

Summary of Changes

Hello @brian-dellabetta, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request updates the AWQModifier to be more flexible regarding activation quantization. Previously, the system would error out if an AWQ scheme included activation quantization that wasn't 16-bit. This change loosens that restriction, allowing such schemes to proceed while providing a clear warning to the user about the experimental nature of lower-precision activation quantization with AWQ and offering guidance for better performance.

Highlights

• AWQ Activation Quantization Policy: The strict assertion that AWQ activations must be 16-bit precision has been replaced with a warning. This allows AWQModifier to be instantiated with schemes including lower-precision activation quantization, resolving issue #1657.
• User Guidance: A warning message is now issued when non-16-bit activation quantization is detected, informing users that this is an experimental feature and suggesting W4A16 or W4A16_ASYM schemes if overall performance is poor.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[gemini-code-assist]

Code Review

This pull request updates the AWQModifier to issue a warning instead of an error when activation quantization is used, which is a sensible change to allow for experimentation. The implementation is straightforward. I've added a couple of minor suggestions to fix a typo in the new warning message and to improve the conciseness of the code.