pytorch-optimizer v2.11.0

Change Log

Feature

• Implement PAdam optimizer (#186)
  • Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
• Implement LOMO optimizer (#188)
  • Full Parameter Fine-tuning for Large Language Models with Limited Resources
• Implement loss functions (#189)
  • BCELoss
  • BCEFocalLoss
  • FocalLoss : Focal Loss for Dense Object Detection
  • FocalCosineLoss : Data-Efficient Deep Learning Method for Image Classification Using Data Augmentation, Focal Cosine Loss, and Ensemble
  • DiceLoss : Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations
  • LDAMLoss : Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
  • JaccardLoss
  • BiTemperedLogisticLoss : Robust Bi-Tempered Logistic Loss Based on Bregman Divergences

Diff

2.10.1...2.11.0