Merge pull request #353 from kozistr/update/schedulefree-optimizers

[Update] ScheduleFree optimizers

Author: kozistr

docs/changelogs/v3.4.2.md

@@ -5,10 +5,20 @@
 * Implement `SCION` optimizer. (#348, #352)
     * [Training Deep Learning Models with Norm-Constrained LMOs](https://arxiv.org/abs/2502.07529)
 
+### Update
+
+* Update ScheduleFreeSGD, AdamW, RAdam optimizers with the latest. (#351, #353)
+* Remove `use_palm` variant in ScheduleFree optimizer due to instability. (#353)
+* Ranger25 optimizer. (#353)
+
+### Fix
+
+* Remove `weight decouple` parameter in ScheduleFree optimizers. (#351, #353)
+
 ### Docs
 
 * Fix `AliG` optimizer visualization. (#350)
 
 ### Contributions
 
-thanks to @AidinHamedi
+thanks to @AidinHamedi, @hatonosuke

docs/visualization.md

@@ -302,6 +302,10 @@
 
 ![image](https://raw.githubusercontent.com/kozistr/pytorch_optimizer/main/docs/visualizations/rastrigin_ScheduleFreeSGD.png)
 
+### SCION
+
+![image](https://raw.githubusercontent.com/kozistr/pytorch_optimizer/main/docs/visualizations/rastrigin_SCION.png)
+
 ### SGD
 
 ![image](https://raw.githubusercontent.com/kozistr/pytorch_optimizer/main/docs/visualizations/rastrigin_SGD.png)
@@ -668,6 +672,10 @@
 
 ![image](https://raw.githubusercontent.com/kozistr/pytorch_optimizer/main/docs/visualizations/rosenbrock_ScheduleFreeSGD.png)
 
+### SCION
+
+![image](https://raw.githubusercontent.com/kozistr/pytorch_optimizer/main/docs/visualizations/rosenbrock_SCION.png)
+
 ### SGD
 
 ![image](https://raw.githubusercontent.com/kozistr/pytorch_optimizer/main/docs/visualizations/rosenbrock_SGD.png)