[Feature] Implement Conda optimizer (#441)

* docs: v3.8.1 changelog

* docs: README

* feature: Conda optimizer

* update: test cases

* update: condition

* update: condition

* [skip ci] docs: update index.md

* docs: Conda optimizer

* chore: keyword

* build(deps): dev deps

* ci: bump pyright to 1.1.406

* update: test_galore_projection_type

* update: test_galore_methods

* update: test_galore_methods

* update: test_galore_methods

---------

Co-authored-by: kozistr <[email protected]>

Author: kozistr

.github/workflows/static_analysis.yml

@@ -24,4 +24,4 @@ jobs:
 
     - uses: jakebailey/pyright-action@v2
       with:
-        version: 1.1.404
+        version: 1.1.406

README.md

@@ -10,7 +10,7 @@
 
 ## The reasons why you use `pytorch-optimizer`.
 
-* Wide range of supported optimizers. Currently, **128 optimizers (+ `bitsandbytes`, `qgalore`, `torchao`)**, **16 lr schedulers**, and **13 loss functions** are supported!
+* Wide range of supported optimizers. Currently, **129 optimizers (+ `bitsandbytes`, `qgalore`, `torchao`)**, **16 lr schedulers**, and **13 loss functions** are supported!
 * Including many variants such as `ADOPT`, `Cautious`, `AdamD`, `StableAdamW`, and `Gradient Centrailiaztion`
 * Easy to use, clean, and tested codes
 * Active maintenance
@@ -222,6 +222,7 @@ get_supported_optimizers(['adam*', 'ranger*'])
 | Refined Schedule-Free | *Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training*            |                                                                                                                | <https://arxiv.org/abs/2507.09846>                                                                                   | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250709846S/exportcitation)                                                        |
 | FriendlySAM           | *Friendly Sharpness-Aware Minimization*                                                                        | [github](https://github.com/nblt/F-SAM)                                                                        | <https://openaccess.thecvf.com/content/CVPR2024/papers/Li_Friendly_Sharpness-Aware_Minimization_CVPR_2024_paper.pdf> | [cite](https://github.com/nblt/F-SAM?tab=readme-ov-file#citation)                                                                   |
 | AdaGO                 | *AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates*                                                |                                                                                                                | <https://arxiv.org/abs/2509.02981>                                                                                   | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250902981Z/exportcitation)                                                        |
+| Conda                 | *Column-Normalized Adam for Training Large Language Models Faster*                                             | [github](https://github.com/jie040109/Conda)                                                                   | <https://arxiv.org/abs/2509.24218>                                                                                   | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250924218W/exportcitation)                                                        |
 
 ## Supported LR Scheduler
 

docs/changelogs/v3.8.1.md

@@ -8,6 +8,8 @@
     * [AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates](https://arxiv.org/abs/2509.02981) 
 * Update `EXAdam` optimizer to the latest version. (#438)
 * Update `EmoNavi` optimizer to the latest version. (#433, #439)
+* Implement `Conda` optimizer. (#440, #441)
+    * [Conda: Column-Normalized Adam for Training Large Language Models Faster](https://arxiv.org/abs/2509.24218)
 
 ### Bug
 

docs/index.md

@@ -10,7 +10,7 @@
 
 ## The reasons why you use `pytorch-optimizer`.
 
-* Wide range of supported optimizers. Currently, **128 optimizers (+ `bitsandbytes`, `qgalore`, `torchao`)**, **16 lr schedulers**, and **13 loss functions** are supported!
+* Wide range of supported optimizers. Currently, **129 optimizers (+ `bitsandbytes`, `qgalore`, `torchao`)**, **16 lr schedulers**, and **13 loss functions** are supported!
 * Including many variants such as `ADOPT`, `Cautious`, `AdamD`, `StableAdamW`, and `Gradient Centrailiaztion`
 * Easy to use, clean, and tested codes
 * Active maintenance
@@ -222,6 +222,7 @@ get_supported_optimizers(['adam*', 'ranger*'])
 | Refined Schedule-Free | *Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training*            |                                                                                                                | <https://arxiv.org/abs/2507.09846>                                                                                   | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250709846S/exportcitation)                                                        |
 | FriendlySAM           | *Friendly Sharpness-Aware Minimization*                                                                        | [github](https://github.com/nblt/F-SAM)                                                                        | <https://openaccess.thecvf.com/content/CVPR2024/papers/Li_Friendly_Sharpness-Aware_Minimization_CVPR_2024_paper.pdf> | [cite](https://github.com/nblt/F-SAM?tab=readme-ov-file#citation)                                                                   |
 | AdaGO                 | *AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates*                                                |                                                                                                                | <https://arxiv.org/abs/2509.02981>                                                                                   | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250902981Z/exportcitation)                                                        |
+| Conda                 | *Column-Normalized Adam for Training Large Language Models Faster*                                             | [github](https://github.com/jie040109/Conda)                                                                   | <https://arxiv.org/abs/2509.24218>                                                                                   | [cite](https://ui.adsabs.harvard.edu/abs/2025arXiv250924218W/exportcitation)                                                        |
 
 ## Supported LR Scheduler
 

docs/optimizer.md

@@ -160,6 +160,10 @@
     :docstring:
     :members:
 
+::: pytorch_optimizer.Conda
+    :docstring:
+    :members:
+
 ::: pytorch_optimizer.DAdaptAdaGrad
     :docstring:
     :members: