Skip to content

[bugfix] Fix metric megatron #7905

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Jintao-Huang merged 5 commits into from
Jan 26, 2026
Merged

[bugfix] Fix metric megatron #7905

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
480182c
fix swift v4 bug
Jintao-Huang Jan 25, 2026
5f30e11
update
Jintao-Huang Jan 25, 2026
b875199
update
Jintao-Huang Jan 25, 2026
aeb2453
fix eval_metric megatron
Jintao-Huang Jan 26, 2026
d0314cb
Merge branch 'main' into fix_metric_megatron
Jintao-Huang Jan 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions swift/megatron/arguments/megatron_args.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -632,6 +632,8 @@ def _set_default(self):
self.norm_epsilon = 1e-5
if self.rotary_base is None:
self.rotary_base = 10000
else:
self.rotary_base = int(self.rotary_base)
Comment on lines +635 to +636
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The else block ensures that self.rotary_base is explicitly converted to an integer if it's not None. This is a good practice to maintain type consistency, especially if rotary_base can be passed as a string from external configurations (e.g., command-line arguments).

if self.rotary_interleaved is None:
self.rotary_interleaved = False
if self.attention_dropout is None:
Expand Down
6 changes: 3 additions & 3 deletions swift/megatron/trainers/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -711,9 +711,9 @@ def evaluate(
total_loss_dict[key] = numerator / denominator
if self.eval_metrics is not None:
metric = self.eval_metrics.compute()
for k, v in metric.items():
total_loss_dict[k] = v if isinstance(v, torch.Tensor) else torch.tensor(v)
self.eval_metrics.reset()
for k, v in metric.items():
total_loss_dict[k] = v if isinstance(v, torch.Tensor) else torch.tensor(v)
self.eval_metrics.reset()
timers('evaluate').stop()
timers.log(['evaluate'])
self.custom_log(total_loss_dict, 'eval')
Expand Down