[bugfix] fix qwen3_5 fp8 gpt-bridge

[Jintao-Huang]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @Jintao-Huang, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a bug related to the qwen3_5 model, specifically concerning fp8 quantization within the gpt-bridge component. It refactors a parameter name for model revisions and enhances the gpt_bridge's ability to correctly interpret and load model states for Mixture-of-Experts (MoE) architectures, ensuring proper functionality and compatibility.

Highlights

• Renamed model revision parameter: The model_revision parameter in the get_model_processor function and its callers has been renamed to revision for improved clarity and consistency across the codebase.
• Enhanced GPT Bridge compatibility for MoE models: The gpt_bridge.py module was updated to correctly handle Mixture-of-Experts (MoE) configurations and state dictionary loading, particularly for qwen3_5 models with fp8 quantization, by refining the logic for hf_grouped and is_gate_up detection using regular expressions.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• swift/arguments/base_args/base_args.py
  • Renamed the model_revision parameter to revision in the get_model_processor function definition.
  • Updated the internal dictionary assignment within get_model_processor to use the new revision key.
• swift/megatron/model/gpt_bridge.py
  • Imported the re module for regular expression operations.
  • Modified the _set_mlp_state function to improve detection of grouped experts and gate-up projections, especially when converting to mcore format, by using regex for pattern matching in state dictionary keys.
  • Adjusted the transformers 5.0 compatibility check to exclude to_mcore scenarios.
• swift/pipelines/train/rlhf.py
  • Updated the call to args.get_model_processor to pass the model_revision value to the new revision parameter.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request addresses a bug in the GPT bridge for Qwen 3.5 FP8 models by improving the detection of grouped experts and gate-up projections through state dictionary inspection. It also unifies the revision argument naming in get_model_processor across the codebase, ensuring that model revisions are correctly passed to the underlying loaders. The changes are well-structured and improve the robustness of weight conversion between HuggingFace and Megatron formats.

[gemini-code-assist]

Using regex to detect separate experts in the state dict is a robust way to determine hf_grouped when loading weights. This is more reliable than relying on hardcoded model lists. You can simplify the generator expression slightly for better readability.

References

1. Iterating directly over a dictionary iterates over its keys, and match objects are truthy in Python.

[gemini-code-assist]

Detecting is_gate_up by checking for the presence of the string in the state dict keys is a good improvement for robustness when to_mcore is True. The .keys() call can be omitted for brevity.

References

1. Iterating directly over a dictionary iterates over its keys.

[Jintao-Huang]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces several bug fixes and improvements, primarily for the qwen3_5 model's fp8 and gpt-bridge functionality. The changes include renaming model_revision to revision for better consistency with Hugging Face Transformers, renaming fp8_param_gather to fp8_param, and significantly refactoring the MoE handling logic in gpt_bridge.py. The updated logic for MoE models is more robust as it now inspects the state dictionary directly to determine expert grouping and layer structure, which is a solid improvement. Overall, the changes are well-implemented and enhance the codebase's correctness and consistency. I have no further comments.