File tree Expand file tree Collapse file tree 1 file changed +8
-0
lines changed Expand file tree Collapse file tree 1 file changed +8
-0
lines changed Original file line number Diff line number Diff line change 1414* Improve the stability of ` ADOPT ` optimizer. (#294 )
1515 * [ Note] ( https://github.com/iShohei220/adopt?tab=readme-ov-file#update-on-nov-22-2024 )
1616* Support a new projection type ` random ` for ` GaLoreProjector ` . (#294 )
17+ * Implement ` DeMo ` optimizer. (#300 , #301 )
18+ * [ Decoupled Momentum Optimization] ( https://arxiv.org/abs/2411.19870 )
19+ * Implement ` Muon ` optimizer. (#302 )
20+ * [ MomentUm Orthogonalized by Newton-schulz] ( https://github.com/KellerJordan/Muon )
21+ * Implement ` ScheduleFreeRAdam ` optimizer. (#304 )
22+ * Implement ` LaProp ` optimizer. (#304 )
23+ * [ Separating Momentum and Adaptivity in Adam] ( https://arxiv.org/abs/2002.04839 )
24+ * Support ` Cautious ` variant to ` LaProp ` , ` AdamP ` , ` Adopt ` optimizers. (#304 ).
1725
1826### Refactor
1927
You can’t perform that action at this time.
0 commit comments