Adaptive learning rate for Muon: NorMuon and AdaMuon#76

Merged

skyw merged 32 commits intoNVIDIA-NeMo:mainfrom

mkhona-nvidia:adaptive_orthogonalized_optimizer

Nov 18, 2025

Commits on Nov 18, 2025

support adaptive learning rate for Muon: normuon and adamuon
mkhona-nvidia
committed
removed adaptive orthogonalized optimizer as separate class, supported second moment computations within same code
mkhona-nvidia
committed
removed extra literal
mkhona-nvidia
committed
subclassed orthogonalized optimizer and override step instead of mixin
mkhona-nvidia
committed
removed mixin test
mkhona-nvidia
committed
cleaned up adaptive orthogonalized optimizer
mkhona-nvidia
committed
changed second moment to moment2, addressed other MR comments
mkhona-nvidia
committed
removed args doc
mkhona-nvidia
committed
made a separate test file for adaptive orthogonalized optimizer
mkhona-nvidia
committed
added device flag
mkhona-nvidia
committed
changed test name
mkhona-nvidia
committed
changed value error to type error
mkhona-nvidia
committed
removed adaptive test
mkhona-nvidia
committed
changed subclass to Muon instead of OrthogonalizedOptimizer
mkhona-nvidia
committed
changed name to adaptivemuon, updated tests
mkhona-nvidia
committed
updated test to import from adaptive_muon
mkhona-nvidia
committed
added missing copyright
mkhona-nvidia
committed
changed scale mode to 1.0, added it as a scale mode
mkhona-nvidia
committed
Revert "changed scale mode to 1.0, added it as a scale mode"
mkhona-nvidia
committed
addressed MR comments
mkhona-nvidia
committed
use consistent orth_grad naming
mkhona-nvidia
committed
addressed MR comment
mkhona-nvidia
committed
addressed MR comments
mkhona-nvidia
committed
addressed MR comments
mkhona-nvidia
committed
added adaptive_muon test to CI
mkhona-nvidia
committed
changed name to adaptive_muon
mkhona-nvidia
committed
changed b2 default to 0.95
mkhona-nvidia
committed
added ref
mkhona-nvidia
committed
Revert "added ref"
mkhona-nvidia
committed
added ref
mkhona-nvidia
committed
fixed test
mkhona-nvidia
committed
raised typereror during correct place
mkhona-nvidia
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Adaptive learning rate for Muon: NorMuon and AdaMuon#76

Adaptive learning rate for Muon: NorMuon and AdaMuon#76
skyw merged 32 commits intoNVIDIA-NeMo:mainfrom
mkhona-nvidia:adaptive_orthogonalized_optimizer

Commits on Nov 18, 2025

support adaptive learning rate for Muon: normuon and adamuon

removed adaptive orthogonalized optimizer as separate class, supported second moment computations within same code

removed extra literal

subclassed orthogonalized optimizer and override step instead of mixin

removed mixin test

cleaned up adaptive orthogonalized optimizer

changed second moment to moment2, addressed other MR comments

removed args doc

made a separate test file for adaptive orthogonalized optimizer

added device flag

changed test name

changed value error to type error

removed adaptive test

changed subclass to Muon instead of OrthogonalizedOptimizer

changed name to adaptivemuon, updated tests

updated test to import from adaptive_muon

added missing copyright

changed scale mode to 1.0, added it as a scale mode

Revert "changed scale mode to 1.0, added it as a scale mode"

addressed MR comments

use consistent orth_grad naming

addressed MR comment

addressed MR comments

addressed MR comments

added adaptive_muon test to CI

changed name to adaptive_muon

changed b2 default to 0.95

added ref

Revert "added ref"

added ref

fixed test

raised typereror during correct place