File tree Expand file tree Collapse file tree 1 file changed +4
-1
lines changed
Expand file tree Collapse file tree 1 file changed +4
-1
lines changed Original file line number Diff line number Diff line change 3232class PSGDPro (torch .optim .Optimizer ):
3333 """Implements a variant of the PSGD optimization algorithm (PSGD-Kron-Whiten with Procrustes step for preconditioner update).
3434
35- PSGD ()
35+ Preconditioned Stochastic Gradient Descent (PSGD) (https://arxiv.org/abs/1512.04202) is a preconditioned optimization algorithm
36+ that fits amplitudes of perturbations of preconditioned stochastic gradient to match that of the perturbations of parameters.
37+ PSGD with Kronecker-factored Preconditioner (PSGD-Kron-Whiten) is a variant of PSGD that reduces memory and computational complexity.
38+ Procrustes step is an algorithm to update the preconditioner which respects a particular geometry.
3639
3740 Args:
3841 params: Iterable of parameters to optimize or dicts defining parameter groups
You can’t perform that action at this time.
0 commit comments