We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 9753eda commit 6b3dd0eCopy full SHA for 6b3dd0e
pytorch_optimizer/optimizer/scion.py
@@ -409,7 +409,7 @@ def step(self, closure: CLOSURE = None) -> LOSS:
409
410
d.mul_(1.0 - group['momentum']).add_(grad, alpha=group['momentum'])
411
412
- update = norm.lmo(grad).mul_(group['scale'])
+ update = norm.lmo(d).mul_(group['scale'])
413
414
if group['constraint']:
415
p.mul_(1.0 - group['lr'])
0 commit comments