pytorch-optimizer v3.7.0

Change Log

Feature

• Implement AdaMuon optimizer. (#394, #395)
  • Adaptive Muon Optimizer
• Implement SPlus optimizer. (#396, #399)
  • A Stable Whitening Optimizer for Efficient Neural Network Training
• Implement EmoNavi, EmoFact, and EmoLynx optimizers. (#393, #400)
  • An emotion-driven optimizer that feels loss and navigates accordingly

CI

• Enable CI for Python 3.8 ~ 3.13. (#402, #404)

Fix

• Adjust the value of eps to the fixed value 1e-15 when adding to exp_avg_sq. (#397, #398)
• built-in type-hint in Kron optimizer. (#404)

Contributions

Thanks to @sobolevn