Conversation
Signed-off-by: limuyuan <limuyuan3@huawei.com>
…the class does not update the already-compiled validation schema, so rebuild_dataclass(..., force=True) is required for instance construction to actually run the patched validator. Signed-off-by: limuyuan <limuyuan3@huawei.com>
…eConfig.__pydantic_decorators__.model_validators[_verify_args].func before rebuild_dataclass, since Pydantic dataclass validation uses the cached Decorator.func rather than the class attribute alone. Signed-off-by: limuyuan <limuyuan3@huawei.com>
…tive_config validation uses the updated schema when VllmConfig was imported before the patch. Signed-off-by: limuyuan <limuyuan3@huawei.com>
Signed-off-by: limuyuan <limuyuan3@huawei.com>
…ses on NPU Signed-off-by: limuyuan <limuyuan3@huawei.com>
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request integrates Eagle3 speculative decoding capabilities with the MiniMax-M2 model within the vLLM framework. It addresses compatibility issues by patching configuration validation to include MiniMax-M2, registering necessary model aliases, and modifying the MiniMax-M2 model's forward pass to expose auxiliary hidden states required by the Eagle3 mechanism. Additionally, it ensures the MiniMax-M2 model adheres to the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive support for MiniMax-M2 models with Eagle3 speculative decoding on NPU. Key changes include patching SpeculativeConfig to allow MiniMax-M2 models in the Eagle3 whitelist, registering Eagle3MiniMaxM2ForCausalLM as an alias in the model registry, and modifying MiniMaxM2Model.forward to collect and return auxiliary hidden states with pipeline parallelism support. Additionally, MiniMaxM2ForCausalLM is updated to conform to the SupportsEagle3 interface, and MiniMaxM2Attention.forward is optimized using a fused kernel. A review comment points out that the documentation for the MiniMaxM2Attention.forward patch is missing from vllm_ascend/patch/__init__.py, which is important for maintainability.
| # 4. `vllm.model_executor.models.minimax_m2.MiniMaxM2Model.forward` | ||
| # Why: | ||
| # Eagle3 speculative decoding needs auxiliary hidden states from specific | ||
| # transformer layers of the target model. | ||
| # How： | ||
| # Extend `MiniMaxM2Model.forward` to optionally collect and return | ||
| # `(final_hidden_states, aux_hidden_states)` when `aux_hidden_state_layers` | ||
| # is set by the runtime. | ||
| # Related PR (if no, explain why): | ||
| # https://github.com/vllm-project/vllm/pull/37512 | ||
| # Future Plan: | ||
| # Remove this patch once upstream MiniMax-M2 integrates Eagle3 support. | ||
| # | ||
| # 5. `vllm.model_executor.models.minimax_m2.MiniMaxM2ForCausalLM` | ||
| # Why: | ||
| # vLLM core uses SupportsEagle3-style methods to configure which layers | ||
| # should emit auxiliary hidden states. | ||
| # How： | ||
| # Inject `set_aux_hidden_state_layers` and default-layer getters onto | ||
| # `MiniMaxM2ForCausalLM` so vLLM can configure the target model. | ||
| # Related PR (if no, explain why): | ||
| # https://github.com/vllm-project/vllm/pull/37512 | ||
| # Future Plan: | ||
| # Remove this patch once upstream provides these methods on the model. | ||
| # |
There was a problem hiding this comment.
The patch for vllm.model_executor.models.minimax_m2.MiniMaxM2Attention.forward in worker/patch_minimax_m2.py is not documented here. This file serves as a manifest for all patches, and for maintainability, it's important to keep it complete and up-to-date. Please add documentation for this new patch. It appears to be a performance optimization using the fused kernel torch.ops.vllm.split_qkv_tp_rmsnorm_rope.
You could add it as item 3 and renumber the subsequent items under ** 17. File: worker/patch_minimax_m2.py**.
MengqingCao
added
ready
2 participants
What this PR does / why we need it?
Apply Eagle3 to MiniMax-M2.5 to increase model performance
This will be discard after Eagle3 weight for MiniMax-M2.5 releases and code change accepted by official repo
https://github.com/vllm-project/vllm/pull/37512/changes
Does this PR introduce any user-facing change?
How was this patch tested?