pytorch-optimizer v3.8.1

Change Log

Feature

• Implement FriendlySAM optimizer. (#424, #434)
  • Friendly Sharpness-Aware Minimization
• Implement AdaGO optimizer. (#436, #437)
  • AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates
• Update EXAdam optimizer to the latest version. (#438)
• Update EmoNavi optimizer to the latest version. (#433, #439)
• Implement Conda optimizer. (#440, #441)
  • Conda: Column-Normalized Adam for Training Large Language Models Faster

Update

• Accept the GaloreProjector parameters in the init params of the Conda optimizer. (#443, #444)

Bug

• Fix NaN problem when grad norm is zero in StableSPAM optimizer. (#431)

Docs

• Update the documentation page. (#428)

Contribution

thanks to @liveck, @AhmedMostafa16