Skip to content

Commit 27d6b99

Browse files
committed
refactor: n_sma_max
1 parent 26fca7b commit 27d6b99

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

pytorch_optimizer/optimizer/adabelief.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,9 @@ def step(self, closure: CLOSURE = None) -> LOSS:
9595

9696
for group in self.param_groups:
9797
beta1, beta2 = group['betas']
98-
n_sma_max: float = 2 / (1 - beta2) - 1
98+
if self.rectify:
99+
n_sma_max: float = 2.0 / (1.0 - beta2) - 1.0
100+
99101
for p in group['params']:
100102
if p.grad is None:
101103
continue

0 commit comments

Comments
 (0)