[Feature] Implement EmoNavi, EmoFact, and EmoLynx optimizers (#400)

* docs: EmoNavi optimizer

* feature: EmoNavi optimizer

* chore: keyword

* update: codes

* docs: v3.6.2 changelog

* fix: alpha to weight

* update: closure

* update: recipes

* fix: closure

* update: loss

* refactor: shadow_weight

Author: kozistr

README.md

@@ -218,6 +218,7 @@ get_supported_optimizers(['adam*', 'ranger*'])
 | AdamC               | *Why Gradients Rapidly Increase Near the End of Training*                                                      |                                                                                                                | <https://arxiv.org/abs/2506.02285>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250602285D/exportcitation)                                                        |
 | AdaMuon             | *Adaptive Muon Optimizer*                                                                                      |                                                                                                                | <https://arxiv.org/abs/2507.11005v1>                                                        | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250711005S/exportcitation)                                                        |
 | SPlus               | *A Stable Whitening Optimizer for Efficient Neural Network Training*                                           | [github](https://github.com/kvfrans/splus)                                                                     | <https://arxiv.org/abs/2506.07254>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250607254F/exportcitation)                                                        |
+| EmoNavi             | *An emotion-driven optimizer that feels loss and navigates accordingly.*                                       | [github](https://github.com/muooon/EmoNavi)                                                                    |                                                                                             |                                                                                                                                     |
 
 ## Supported LR Scheduler
 

docs/changelogs/v3.6.2.md

@@ -6,6 +6,8 @@
     * [Adaptive Muon Optimizer](https://arxiv.org/abs/2507.11005v1)
 * Implement `SPlus` optimizer. (#396, #399)
     * [A Stable Whitening Optimizer for Efficient Neural Network Training](https://arxiv.org/abs/2506.07254) 
+* Implement `EmoNavi`, `EmoFact`, and `EmoLynx` optimizers. (#393, #400)
+    * [An emotion-driven optimizer that feels loss and navigates accordingly](https://github.com/muooon/EmoNavi)
 
 ### Fix
 

docs/index.md

@@ -218,6 +218,7 @@ get_supported_optimizers(['adam*', 'ranger*'])
 | AdamC               | *Why Gradients Rapidly Increase Near the End of Training*                                                      |                                                                                                                | <https://arxiv.org/abs/2506.02285>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250602285D/exportcitation)                                                        |
 | AdaMuon             | *Adaptive Muon Optimizer*                                                                                      |                                                                                                                | <https://arxiv.org/abs/2507.11005v1>                                                        | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250711005S/exportcitation)                                                        |
 | SPlus               | *A Stable Whitening Optimizer for Efficient Neural Network Training*                                           | [github](https://github.com/kvfrans/splus)                                                                     | <https://arxiv.org/abs/2506.07254>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250607254F/exportcitation)                                                        |
+| EmoNavi             | *An emotion-driven optimizer that feels loss and navigates accordingly.*                                       | [github](https://github.com/muooon/EmoNavi)                                                                    |                                                                                             |                                                                                                                                     |
 
 ## Supported LR Scheduler
 

docs/optimizer.md

@@ -184,6 +184,18 @@
     :docstring:
     :members:
 
+::: pytorch_optimizer.EmoFact
+    :docstring:
+    :members:
+
+::: pytorch_optimizer.EmoLynx
+    :docstring:
+    :members:
+
+::: pytorch_optimizer.EmoNavi
+    :docstring:
+    :members:
+
 ::: pytorch_optimizer.EXAdam
     :docstring:
     :members:

pyproject.toml

@@ -14,15 +14,15 @@ keywords = [
     "AdaBound", "AdaDelta", "AdaFactor", "AdaGC", "AdaMax", "AdaMuon", "AdamG", "AdaMod", "AdaNorm", "AdaPNM",
     "AdaSmooth", "AdEMAMix", "Simplified-AdEMAMix", "ADOPT", "AdaHessian", "Adai", "Adalite", "AdaLomo", "AdamMini",
     "AdamP", "AdamS", "Adan", "AggMo", "Aida", "AliG", "Amos", "Apollo", "APOLLO", "AvaGrad", "bSAM", "CAME",
-    "DAdaptAdaGrad", "DAdaptAdam", "DAdaptAdan", "DAdaptSGD", "DAdaptLion", "DeMo", "DiffGrad", "EXAdam", "FAdam",
-    "Fira", "FOCUS", "Fromage", "FTRL", "GaLore", "Grams", "Gravity", "GrokFast", "GSAM", "Kate", "Lamb", "LaProp",
-    "LARS", "Lion", "LOMO", "Lookahead", "MADGRAD", "MARS", "MSVAG", "Muno", "Nero", "NovoGrad", "OrthoGrad", "PAdam",
-    "PCGrad", "PID", "PNM", "Prodigy", "PSGD", "QHAdam", "QHM", "RACS", "RAdam", "Ranger", "Ranger21", "RotoGrad",
-    "SAM", "GCSAM", "LookSAM", "ScheduleFreeSGD", "ScheduleFreeAdamW", "ScheduleFreeRAdam", "SCION", "SGDP", "Shampoo",
-    "ScalableShampoo", "SGDW", "SignSGD", "SM3", "SOAP", "SopihaH", "SPAM", "StableSPAM", "SPlus", "SRMM",
-    "StableAdamW", "SWATS", "TAM", "Tiger", "TRAC", "VSGD", "WSAM", "Yogi", "BCE", "BCEFocal", "Focal", "FocalCosine",
-    "SoftF1", "Dice", "LDAM", "Jaccard", "Bi-Tempered", "Tversky", "FocalTversky", "LovaszHinge", "bitsandbytes", "WSD",
-    "QGaLore",
+    "DAdaptAdaGrad", "DAdaptAdam", "DAdaptAdan", "DAdaptSGD", "DAdaptLion", "DeMo", "DiffGrad", "EmoFact", "EmoLynx",
+    "EmoNavi", "EXAdam", "FAdam", "Fira", "FOCUS", "Fromage", "FTRL", "GaLore", "Grams", "Gravity", "GrokFast", "GSAM",
+    "Kate", "Lamb", "LaProp", "LARS", "Lion", "LOMO", "Lookahead", "MADGRAD", "MARS", "MSVAG", "Muno", "Nero",
+    "NovoGrad", "OrthoGrad", "PAdam", "PCGrad", "PID", "PNM", "Prodigy", "PSGD", "QHAdam", "QHM", "RACS", "RAdam",
+    "Ranger", "Ranger21", "RotoGrad", "SAM", "GCSAM", "LookSAM", "ScheduleFreeSGD", "ScheduleFreeAdamW",
+    "ScheduleFreeRAdam", "SCION", "SGDP", "Shampoo", "ScalableShampoo", "SGDW", "SignSGD", "SM3", "SOAP", "SopihaH",
+    "SPAM", "StableSPAM", "SPlus", "SRMM", "StableAdamW", "SWATS", "TAM", "Tiger", "TRAC", "VSGD", "WSAM", "Yogi",
+    "BCE", "BCEFocal", "Focal", "FocalCosine", "SoftF1", "Dice", "LDAM", "Jaccard", "Bi-Tempered", "Tversky",
+    "FocalTversky", "LovaszHinge", "bitsandbytes", "WSD", "QGaLore",
 ]
 classifiers = [
     "License :: OSI Approved :: Apache Software License",

pytorch_optimizer/__init__.py

@@ -114,6 +114,9 @@
     DeMo,
     DiffGrad,
     DynamicLossScaler,
+    EmoFact,
+    EmoLynx,
+    EmoNavi,
     EXAdam,
     FAdam,
     Fira,

pytorch_optimizer/optimizer/__init__.py

@@ -43,6 +43,7 @@
 from pytorch_optimizer.optimizer.dadapt import DAdaptAdaGrad, DAdaptAdam, DAdaptAdan, DAdaptLion, DAdaptSGD
 from pytorch_optimizer.optimizer.demo import DeMo
 from pytorch_optimizer.optimizer.diffgrad import DiffGrad
+from pytorch_optimizer.optimizer.emonavi import EmoFact, EmoLynx, EmoNavi
 from pytorch_optimizer.optimizer.exadam import EXAdam
 from pytorch_optimizer.optimizer.experimental.ranger25 import Ranger25
 from pytorch_optimizer.optimizer.fadam import FAdam
@@ -325,6 +326,9 @@ def load_optimizer(optimizer: str) -> OPTIMIZER:
     VSGD,
     AdaMuon,
     SPlus,
+    EmoFact,
+    EmoLynx,
+    EmoNavi,
 ]
 OPTIMIZERS: Dict[str, OPTIMIZER] = {str(optimizer.__name__).lower(): optimizer for optimizer in OPTIMIZER_LIST}