pytorch-optimizer v2.4.1

Change Log

Feature

• Rename the new Shampoo to ScalableShampoo. #103
• Implement the old(?) version of Shampoo optimizer. #103
• Support SVD method to calculate the inverse pth root matrix. #103
  • to boost the M^{-1/p} calculation, performs batched SVD when available.
• Implement AdamS optimizer. #102
• Support stable weight decay option for Adai optimizer. #102

Bug

• Fix compute_power_svd() to get a singular value. #104