Br vllm ascend 0 18 0 all

[wangyao-i]

What this PR does / why we need it?

Does this PR introduce any user-facing change?

How was this patch tested?

• vLLM version: v0.18.0
• vLLM main: vllm-project/vllm@ed359c4

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors and enhances the Mixture-of-Experts (MoE) gating and MXFP8 quantization scale handling within the vllm_ascend codebase. It introduces a new DeviceOperator class to centralize device-specific operations, ensuring consistent behavior for MoE top-k selection and MXFP8 scale normalization. Additionally, it improves the load balancing strategy for MoE experts, contributing to a more robust and maintainable implementation of quantization and MoE features.

Highlights

• Centralized MoE Gating: Introduced DeviceOperator.moe_gating_top_k to encapsulate and standardize the Mixture-of-Experts (MoE) gating logic, including handling topk_ids type conversion and conditional renormalization.
• Standardized MXFP8 Scale Normalization: Added DeviceOperator.normalize_mxfp8_scale_layout to provide a consistent way to reshape and handle MXFP8 quantization scales across different operations.
• Improved MoE Load Balancing: Updated the expert load balancing mechanism in w8a8_mxfp8.py to use a more robust random selection approach instead of simple random integers, aiming for better expert distribution.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

What this PR does / why we need it?

This pull request introduces a DeviceOperator class to centralize device-specific operations, particularly for Mixture-of-Experts (MoE) gating and MXFP8 quantization scale normalization.

Key changes include:

• Addition of normalize_mxfp8_scale_layout and moe_gating_top_k static methods to DeviceOperator to encapsulate logic for MXFP8 scale handling and MoE gating, respectively.
• Integration of these new DeviceOperator methods into npu_dynamic_quant, npu_grouped_matmul_swiglu_quant, experts_selector.py, and moe_mlp.py to ensure consistent and centralized handling of these operations.
• Refinement of the enable_force_load_balance mechanism in w8a8_mxfp8.py to use a more sophisticated random expert selection approach, improving load balancing for profile runs.

This refactoring aims to improve code organization, maintainability, and consistency across different quantization and MoE related operations.

Fixes #

Does this PR introduce any user-facing change?

No, this PR primarily involves internal refactoring and optimization of device operations and quantization logic. The external API and user-facing behavior should remain unchanged.

How was this patch tested?

CI passed with existing tests. No new tests were added as the changes are internal refactoring and optimization, and existing tests should cover the functionality.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.