Skip to content

Comments

[megatron] update megatron_swift parameter#8077

Merged
Jintao-Huang merged 2 commits intomodelscope:mainfrom
Jintao-Huang:update_megatron_swift_parameter
Feb 23, 2026
Merged

[megatron] update megatron_swift parameter#8077
Jintao-Huang merged 2 commits intomodelscope:mainfrom
Jintao-Huang:update_megatron_swift_parameter

Conversation

@Jintao-Huang
Copy link
Collaborator

No description provided.

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @Jintao-Huang, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors the parameter naming convention within the Megatron-SWIFT framework, transitioning from _interval to _steps for parameters governing training, evaluation, logging, and garbage collection frequencies. This change enhances the semantic clarity of these parameters, making the codebase and documentation more intuitive for users and developers. The updates span core argument definitions, documentation, and numerous example scripts, ensuring a consistent and improved user experience.

Highlights

  • Parameter Renaming: Standardized parameter names in Megatron-SWIFT by renaming *_interval arguments to *_steps for consistency and clarity across various functionalities.
  • Documentation Updates: Updated all relevant documentation files (both Chinese and English versions) to reflect the new save_steps, eval_steps, logging_steps, and manual_gc_steps parameter names.
  • Example Script Adjustments: Modified numerous example shell scripts to use the new save_steps, eval_steps, and logging_steps parameters, ensuring examples align with the updated naming convention.
  • Core Argument Definition Update: Updated the MegatronArguments class in swift/megatron/arguments/megatron_args.py to define and handle the new *_steps parameters, including their default values and interactions.
  • Callback Logic Refinement: Adjusted the DefaultMegatronFlowCallback in swift/megatron/callbacks/default_flow.py to correctly utilize the new logging_steps, eval_steps, save_steps, and manual_gc_steps for controlling training flow.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • docs/source/BestPractices/Qwen3-Best-Practice.md
    • Updated eval_interval to eval_steps and save_interval to save_steps in training commands.
  • docs/source/BestPractices/Qwen3-VL-Best-Practice.md
    • Updated eval_interval to eval_steps and save_interval to save_steps in training commands.
  • docs/source/Instruction/Frequently-asked-questions.md
    • Removed specific mention of save_interval not being supported in Megatron Swift in Q30.
    • Updated save_interval to save_steps in an example script.
  • docs/source/Megatron-SWIFT/Command-line-parameters.md
    • Renamed manual_gc_interval to manual_gc_steps.
    • Renamed save_interval to save_steps.
    • Renamed log_interval to logging_steps.
    • Renamed eval_interval to eval_steps.
    • Updated the description for save_strategy to reference save_steps and eval_steps.
    • Corrected the formatting for generation_batch_size description.
  • docs/source/Megatron-SWIFT/LoRA-Training.md
    • Updated save_interval to save_steps in example training commands.
  • docs/source/Megatron-SWIFT/Mcore-Bridge.md
    • Updated save_interval to save_steps and eval_interval to eval_steps in example training commands.
  • docs/source/Megatron-SWIFT/Multimodal-Model.md
    • Updated save_interval to save_steps and eval_interval to eval_steps in example training commands.
  • docs/source/Megatron-SWIFT/Quick-start.md
    • Updated save_interval to save_steps in example training commands.
  • docs/source_en/BestPractices/Qwen3-Best-Practice.md
    • Updated eval_interval to eval_steps and save_interval to save_steps in training commands.
  • docs/source_en/BestPractices/Qwen3-VL-Best-Practice.md
    • Updated eval_interval to eval_steps and save_interval to save_steps in training commands.
  • docs/source_en/Instruction/Frequently-asked-questions.md
    • Removed specific mention of save_interval not being supported in Megatron-SWIFT in Q30.
    • Updated save_interval to save_steps in an example script.
  • docs/source_en/Megatron-SWIFT/Command-line-parameters.md
    • Renamed manual_gc_interval to manual_gc_steps.
    • Renamed save_interval to save_steps.
    • Renamed log_interval to logging_steps.
    • Renamed eval_interval to eval_steps.
    • Updated the description for save_strategy to reference save_steps and eval_steps.
  • docs/source_en/Megatron-SWIFT/LoRA-Training.md
    • Updated save_interval to save_steps in example training commands.
  • docs/source_en/Megatron-SWIFT/Mcore-Bridge.md
    • Updated save_interval to save_steps and eval_interval to eval_steps in example training commands.
  • docs/source_en/Megatron-SWIFT/Multimodal-Model.md
    • Updated save_interval to save_steps and eval_interval to eval_steps in example training commands.
  • docs/source_en/Megatron-SWIFT/Quick-start.md
    • Updated save_interval to save_steps in example training commands.
  • examples/ascend/multi-node/megatron/node1.sh
    • Updated log_interval to logging_steps.
  • examples/ascend/multi-node/megatron/node2.sh
    • Updated log_interval to logging_steps.
  • examples/ascend/train/qwen3/qwen3_lora_megatron.sh
    • Updated save_interval to save_steps.
  • examples/ascend/train/qwen3_next/qwen3_next_megatron.sh
    • Updated save_interval to save_steps.
  • examples/ascend/train/qwen3_omni/qwen3_omni_full_mindspeed.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/ascend/train/qwen3_vl/moe_full_mindspeed.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/base_to_chat.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/dense/72b_offload.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/dense/qwen3_32b.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/embedding/qwen3_emb.sh
    • Updated save_interval to save_steps and eval_interval to eval_steps.
  • examples/megatron/embedding/qwen3_vl_emb.sh
    • Updated save_interval to save_steps and eval_interval to eval_steps.
  • examples/megatron/fp8/benchmark.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/fp8/llm.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/fp8/vlm.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/grpo/dense_colocate.sh
    • Updated log_interval to logging_steps.
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/grpo/dense_server.sh
    • Updated log_interval to logging_steps.
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/grpo/moe_colocate_full.sh
    • Updated log_interval to logging_steps.
  • examples/megatron/grpo/moe_colocate_lora.sh
    • Updated log_interval to logging_steps.
  • examples/megatron/grpo/sapo.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
    • Updated log_interval to logging_steps.
  • examples/megatron/long_text.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/lora/dense.sh
    • Updated save_interval to save_steps.
  • examples/megatron/lora/dpo.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/lora/loss_scale.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/lora/moe.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/lora/mtp.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/lora/new_special_tokens.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/lora/qwen3_235b.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/mcore_bridge/full/dense.sh
    • Updated save_interval to save_steps.
  • examples/megatron/mcore_bridge/full/moe.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/mcore_bridge/lora/moe.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/mcore_bridge/lora/new_special_tokens.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/mcore_bridge/lora/seq_cls.sh
    • Updated save_interval to save_steps.
  • examples/megatron/moe/deepseek_v3.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/moe/moe.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/moe/qwen3_moe.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/moe/qwen3_moe_offload.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/multi-node/node1.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/multi-node/node2.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/multimodal/dense/dpo.sh
    • Updated save_interval to save_steps.
  • examples/megatron/multimodal/dense/full.sh
    • Updated save_interval to save_steps.
  • examples/megatron/multimodal/dense/lora.sh
    • Updated save_interval to save_steps.
  • examples/megatron/multimodal/moe/full_dpo_offload.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/multimodal/moe/lora.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/multimodal/omni/dense.sh
    • Updated save_interval to save_steps.
  • examples/megatron/multimodal/omni/moe.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/pretrain.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/reranker/qwen3_reranker.sh
    • Updated save_interval to save_steps and eval_interval to eval_steps.
  • examples/megatron/reranker/qwen3_vl_reranker.sh
    • Updated save_interval to save_steps and eval_interval to eval_steps.
  • examples/megatron/rlhf/dpo/dense.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/rlhf/dpo/group_by_length.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/rlhf/dpo/moe.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/rlhf/dpo/packing.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/rlhf/gkd/dense.sh
    • Updated log_interval to logging_steps.
  • examples/megatron/rlhf/kto/dense.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/rlhf/kto/moe.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/rlhf/rm/dense.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/rlhf/rm/moe.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/seq_cls/full.sh
    • Updated save_interval to save_steps and eval_interval to eval_steps.
  • examples/megatron/seq_cls/lora/train.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/megatron/sft.sh
    • Updated save_interval to save_steps.
  • examples/models/glm-4.6v/flash.sh
    • Updated save_interval to save_steps.
  • examples/models/glm-4.6v/mcore.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/models/gpt_oss/internvl3_5_gpt.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/models/gpt_oss/mcore.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/models/llama4/mcore.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/models/qwen3_next/mcore.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/models/qwen3_next/mtp.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/models/qwen3_next/unfused.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/models/qwen3_vl/mcore.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/models/qwen3_vl/mcore_full.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/train/cached_dataset/dpo.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/train/cached_dataset/mcore.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • examples/train/flash_attention_3/mcore.sh
    • Updated eval_interval to eval_steps and save_interval to save_steps.
  • swift/megatron/arguments/megatron_args.py
    • Renamed manual_gc_interval to manual_gc_steps.
    • Renamed save_interval to save_steps.
    • Renamed log_interval to logging_steps.
    • Renamed eval_interval to eval_steps.
    • Updated internal logic to use eval_steps and save_steps for default assignment and save_strategy.
  • swift/megatron/callbacks/default_flow.py
    • Updated conditional checks to use logging_steps, eval_steps, save_steps, and manual_gc_steps.
  • tests/megatron/test_embedding.py
    • Updated save_interval to save_steps in test arguments.
  • tests/megatron/test_gkd.py
    • Updated log_interval to logging_steps in test arguments.
  • tests/megatron/test_grpo.py
    • Updated log_interval to logging_steps.
    • Updated eval_interval to eval_steps and save_interval to save_steps in test arguments.
  • tests/megatron/test_kto.py
    • Updated eval_interval to eval_steps, save_interval to save_steps, and log_interval to logging_steps in test arguments.
  • tests/megatron/test_lora.py
    • Updated save_interval to save_steps in test arguments.
  • tests/megatron/test_rlhf.py
    • Updated eval_interval to eval_steps and log_interval to logging_steps in test arguments.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a consistent naming convention for several training parameters, such as renaming save_interval to save_steps and log_interval to logging_steps. The changes are applied thoroughly across the codebase, including argument definitions, callback logic, documentation, and example scripts. This is a good refactoring that improves clarity and consistency with other popular frameworks. My review includes a couple of suggestions to further improve the clarity of the documentation regarding the save_strategy parameter.

@Jintao-Huang Jintao-Huang merged commit 9204c55 into modelscope:main Feb 23, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants