File tree Expand file tree Collapse file tree 1 file changed +1
-1
lines changed
emerging_optimizers/orthogonalized_optimizers Expand file tree Collapse file tree 1 file changed +1
-1
lines changed Original file line number Diff line number Diff line change @@ -61,7 +61,7 @@ class Muon(OrthogonalizedOptimizer):
6161 ["simple", "quintic", "polar_express"].
6262 num_ns_steps: The number of iteration steps to use in the Newton-Schulz iteration.
6363 scale_mode: The type of scale factor to use for the update. Defaults to "spectral" style scaling.
64- extra_scale_factor: The additional scale factor to use for the update. Set it to 0.2 can closely match
64+ extra_scale_factor: The additional scale factor to use for the update. Setting it to 0.2 can closely match
6565 the update RMS norm of AdamW as suggested by https://arxiv.org/abs/2502.16982.
6666 use_syrk: Whether to use the Triton kernel for the Newton-Schulz iteration.
6767 """
You can’t perform that action at this time.
0 commit comments