Skip to content

Commit b724bb7

Browse files
authored
default use_nesterov to false for Muon (#65)
Signed-off-by: Hao Wu <[email protected]>
1 parent d6b8fa6 commit b724bb7

File tree

1 file changed

+1
-1
lines changed
  • emerging_optimizers/orthogonalized_optimizers

1 file changed

+1
-1
lines changed

emerging_optimizers/orthogonalized_optimizers/muon.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ def __init__(
6464
params: ParamsT,
6565
lr: float = 3e-4,
6666
momentum_beta: float = 0.95,
67-
use_nesterov: bool = True,
67+
use_nesterov: bool = False,
6868
weight_decay: float = 0.01,
6969
use_decoupled_weight_decay: bool = True,
7070
fp32_matmul_prec: str = "medium",

0 commit comments

Comments
 (0)