[Feature] Implement SPlus optimizer (#399)

* docs: SPlus optimizer

* feature: SPlus optimizer

* docs: v3.6.2 changelog

* update: SPlus optimizer

* update: test cases

* build(deps): update dev deps

* chore: keyword

* docs: SPlus optimizer

* docs: README

Author: kozistr

README.md

@@ -10,7 +10,7 @@
 
 ## The reasons why you use `pytorch-optimizer`.
 
-* Wide range of supported optimizers. Currently, **109 optimizers (+ `bitsandbytes`, `qgalore`, `torchao`)**, **16 lr schedulers**, and **13 loss functions** are supported!
+* Wide range of supported optimizers. Currently, **110 optimizers (+ `bitsandbytes`, `qgalore`, `torchao`)**, **16 lr schedulers**, and **13 loss functions** are supported!
 * Including many variants such as `ADOPT`, `Cautious`, `AdamD`, `StableAdamW`, and `Gradient Centrailiaztion`
 * Easy to use, clean, and tested codes
 * Active maintenance
@@ -217,6 +217,7 @@ get_supported_optimizers(['adam*', 'ranger*'])
 | SNSM                | *Subset-Norm and Subspace-Momentum: Faster Memory-Efficient Adaptive Optimization with Convergence Guarantees* | [github](https://github.com/timmytonga/sn-sm)                                                                  | <https://arxiv.org/abs/2411.07120>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2024arXiv241107120N/exportcitation)                                                        |
 | AdamC               | *Why Gradients Rapidly Increase Near the End of Training*                                                      |                                                                                                                | <https://arxiv.org/abs/2506.02285>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250602285D/exportcitation)                                                        |
 | AdaMuon             | *Adaptive Muon Optimizer*                                                                                      |                                                                                                                | <https://arxiv.org/abs/2507.11005v1>                                                        | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250711005S/exportcitation)                                                        |
+| SPlus               | *A Stable Whitening Optimizer for Efficient Neural Network Training*                                           | [github](https://github.com/kvfrans/splus)                                                                     | <https://arxiv.org/abs/2506.07254>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250607254F/exportcitation)                                                        |
 
 ## Supported LR Scheduler
 

docs/changelogs/v3.6.2.md

@@ -4,6 +4,8 @@
 
 * Implement `AdaMuon` optimizer. (#394, #395)
     * [Adaptive Muon Optimizer](https://arxiv.org/abs/2507.11005v1)
+* Implement `SPlus` optimizer. (#396, #399)
+    * [A Stable Whitening Optimizer for Efficient Neural Network Training](https://arxiv.org/abs/2506.07254) 
 
 ### Fix
 

docs/index.md

@@ -10,7 +10,7 @@
 
 ## The reasons why you use `pytorch-optimizer`.
 
-* Wide range of supported optimizers. Currently, **108 optimizers (+ `bitsandbytes`, `qgalore`, `torchao`)**, **16 lr schedulers**, and **13 loss functions** are supported!
+* Wide range of supported optimizers. Currently, **110 optimizers (+ `bitsandbytes`, `qgalore`, `torchao`)**, **16 lr schedulers**, and **13 loss functions** are supported!
 * Including many variants such as `ADOPT`, `Cautious`, `AdamD`, `StableAdamW`, and `Gradient Centrailiaztion`
 * Easy to use, clean, and tested codes
 * Active maintenance
@@ -215,7 +215,9 @@ get_supported_optimizers(['adam*', 'ranger*'])
 | RACS & Alice        | *Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension*     |                                                                                                                | <https://arxiv.org/pdf/2502.07752>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250207752G/exportcitation)                                                        |
 | VSGD                | *Variational Stochastic Gradient Descent for Deep Neural Networks*                                             | [github](https://github.com/generativeai-tue/vsgd)                                                             | <https://openreview.net/forum?id=xu4ATNjcdy>                                                | [cite](https://github.com/generativeai-tue/vsgd/tree/main?tab=readme-ov-file#cite)                                                  |
 | SNSM                | *Subset-Norm and Subspace-Momentum: Faster Memory-Efficient Adaptive Optimization with Convergence Guarantees* | [github](https://github.com/timmytonga/sn-sm)                                                                  | <https://arxiv.org/abs/2411.07120>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2024arXiv241107120N/exportcitation)                                                        |
-| AdamC               | Why Gradients Rapidly Increase Near the End of Training*                                                       |                                                                                                                | <https://arxiv.org/abs/2506.02285>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250602285D/exportcitation)                                                        |
+| AdamC               | *Why Gradients Rapidly Increase Near the End of Training*                                                      |                                                                                                                | <https://arxiv.org/abs/2506.02285>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250602285D/exportcitation)                                                        |
+| AdaMuon             | *Adaptive Muon Optimizer*                                                                                      |                                                                                                                | <https://arxiv.org/abs/2507.11005v1>                                                        | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250711005S/exportcitation)                                                        |
+| SPlus               | *A Stable Whitening Optimizer for Efficient Neural Network Training*                                           | [github](https://github.com/kvfrans/splus)                                                                     | <https://arxiv.org/abs/2506.07254>                                                          | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250607254F/exportcitation)                                                        |
 
 ## Supported LR Scheduler
 

docs/optimizer.md

@@ -432,6 +432,10 @@
     :docstring:
     :members:
 
+::: pytorch_optimizer.SPlus
+    :docstring:
+    :members:
+
 ::: pytorch_optimizer.SRMM
     :docstring:
     :members: