Skip to content

Commit 36aa843

Browse files
committed
updated docstring of psgd
Signed-off-by: mikail <mkhona@nvidia.com>
1 parent 40f33d2 commit 36aa843

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

emerging_optimizers/psgd/psgd.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,10 @@
3232
class PSGDPro(torch.optim.Optimizer):
3333
"""Implements a variant of the PSGD optimization algorithm (PSGD-Kron-Whiten with Procrustes step for preconditioner update).
3434
35-
PSGD ()
35+
Preconditioned Stochastic Gradient Descent (PSGD) (https://arxiv.org/abs/1512.04202) is a preconditioned optimization algorithm
36+
that fits amplitudes of perturbations of preconditioned stochastic gradient to match that of the perturbations of parameters.
37+
PSGD with Kronecker-factored Preconditioner (PSGD-Kron-Whiten) is a variant of PSGD that reduces memory and computational complexity.
38+
Procrustes step is an algorithm to update the preconditioner which respects a particular geometry.
3639
3740
Args:
3841
params: Iterable of parameters to optimize or dicts defining parameter groups

0 commit comments

Comments
 (0)