Skip to content

Commit d4e7564

Browse files
authored
Merge pull request #369 from kozistr/update/scion-optimizer
[Update] Scion and ScionLight optimizers
2 parents 6397d56 + 39fd43e commit d4e7564

26 files changed

+529
-98
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -206,7 +206,7 @@ get_supported_optimizers(['adam*', 'ranger*'])
206206
| EXAdam | *The Power of Adaptive Cross-Moments* | [github](https://github.com/AhmedMostafa16/EXAdam) | <https://arxiv.org/abs/2412.20302> | [cite](https://github.com/AhmedMostafa16/EXAdam?tab=readme-ov-file#citation) |
207207
| GCSAM | *Gradient Centralized Sharpness Aware Minimization* | [github](https://github.com/mhassann22/GCSAM) | <https://arxiv.org/abs/2501.11584> | [cite](https://github.com/mhassann22/GCSAM?tab=readme-ov-file#citation) |
208208
| LookSAM | *Towards Efficient and Scalable Sharpness-Aware Minimization* | [github](https://github.com/rollovd/LookSAM) | <https://arxiv.org/abs/2203.02714> | [cite](https://ui.adsabs.harvard.edu/abs/2022arXiv220302714L/exportcitation) |
209-
| SCION | *Training Deep Learning Models with Norm-Constrained LMOs* | | <https://arxiv.org/abs/2502.07529> | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250207529P/exportcitation) |
209+
| SCION | *Training Deep Learning Models with Norm-Constrained LMOs* | [github](https://github.com/LIONS-EPFL/scion) | <https://arxiv.org/abs/2502.07529> | [cite](https://github.com/LIONS-EPFL/scion?tab=readme-ov-file#citation) |
210210
| COSMOS | *SOAP with Muon* | [github](https://github.com/lliu606/COSMOS) | | |
211211
| StableSPAM | *How to Train in 4-Bit More Stably than 16-Bit Adam* | [github](https://github.com/TianjinYellow/StableSPAM) | <https://arxiv.org/abs/2502.17055> | |
212212
| AdaGC | *Improving Training Stability for Large Language Model Pretraining* | | <https://arxiv.org/abs/2502.11034> | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250211034W/exportcitation) |

docs/changelogs/v3.3.0.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
### Change Log
1+
## Change Log
22

33
### Feature
44

docs/changelogs/v3.3.1.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
### Change Log
1+
## Change Log
22

33
### Feature
44

docs/changelogs/v3.3.2.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
### Change Log
1+
## Change Log
22

33
### Feature
44

docs/changelogs/v3.3.3.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
### Change Log
1+
## Change Log
22

33
### Feature
44

docs/changelogs/v3.3.4.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
### Change Log
1+
## Change Log
22

33
### Feature
44

docs/changelogs/v3.4.0.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
### Change Log
1+
## Change Log
22

33
### Feature
44

docs/changelogs/v3.4.1.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
### Change Log
1+
## Change Log
22

33
### Feature
44

docs/changelogs/v3.4.2.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
### Change Log
1+
## Change Log
22

33
### Feature
44

docs/changelogs/v3.5.0.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
### Change Log
1+
## Change Log
22

33
### Feature
44

0 commit comments

Comments
 (0)