quick fix for w8a8 registry

[shihaobai]

No description provided.

[gemini-code-assist]

Hello @shihaobai, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello! Gemini here, providing a summary of this pull request to help everyone get up to speed quickly. This PR, titled "quick fix for w8a8 registry", appears to address a minor but important issue related to the registration of quantization methods, specifically for the w8a8 (weight 8-bit, activation 8-bit) type. The core change involves correcting the syntax used in a decorator to ensure the method is registered correctly under its intended names.

Highlights

• Fix Quantization Registration: The primary goal of this PR is to fix an incorrect list syntax used in the @QUANTMETHODS.register decorator for the FP8w8a8B128QuantizationMethod class. This ensures the method is properly registered under its two intended keys.

Changelog

• lightllm/common/quantization/w8a8_quant.py
  • Corrected the list syntax in the @QUANTMETHODS.register decorator on line 117, changing ["vllm-fp8w8a8-b128, fp8w8a8-b128"] to ["vllm-fp8w8a8-b128", "fp8w8a8-b128"] to register the method under two distinct names instead of one combined string.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request provides a "quick fix" for the w8a8 registry, specifically correcting how the FP8w8a8B128QuantizationMethod is registered. The change is clear, concise, and directly addresses a likely bug in the previous registration logic. By splitting the single, comma-containing string into two distinct registration keys, this method should now be correctly discoverable under both intended aliases. Well done on identifying and fixing this!

Summary of Findings

• Correctness of Quantization Method Registration: The pull request corrects the registration of FP8w8a8B128QuantizationMethod. Previously, it was likely registered under a single, incorrect key ("vllm-fp8w8a8-b128, fp8w8a8-b128"). The fix changes this to correctly use two distinct keys ("vllm-fp8w8a8-b128" and "fp8w8a8-b128"), ensuring the method is discoverable as intended. This is a crucial fix for the functionality relying on this registration.

Merge Readiness

This pull request appears to be a straightforward and important bug fix. The change correctly registers the FP8w8a8B128QuantizationMethod under two distinct aliases, which was likely the original intent, resolving a potential issue where the method might not have been discoverable. The code is clear and the fix is precise. Based on this review, the change seems ready for merging. However, as an AI reviewer, I am not authorized to approve pull requests. Please ensure further necessary reviews and testing are completed by human reviewers before merging.