Scion optimizer #70

Merged

skyw merged 13 commits intoNVIDIA-NeMo:mainfrom

mkhona-nvidia:mkhona/scion

Nov 5, 2025

Commits on Nov 4, 2025

added Scion's franke-wolfe parametrization to orthogonalized Optimizer
mkhona-nvidia
committed
Revert "added Scion's franke-wolfe parametrization to orthogonalized Optimizer"
mkhona-nvidia
committed
added Scion's franke-wolfe parametrization to orthogonalized Optimizer
mkhona-nvidia
committed
removed choice of scale since we always use unit RMS scaling for muP
mkhona-nvidia
committed
changed named to scion from muon
mkhona-nvidia
committed
changed full form of name to match the paper https://arxiv.org/pdf/2502.07529
mkhona-nvidia
committed
removed wd args from scion
mkhona-nvidia
committed