Commit 82a5f32
authored
feat(pt): add AdaMuon optimizer (#5130)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added AdaMuon optimizer for PyTorch training with mixed 2D/1D
parameter handling, batched orthogonalized updates, momentum and
per-parameter state.
* **Configuration**
* New optimizer options: momentum, weight decay, Adam betas, lr
adjustment modes and tuning coefficients; AdaMuon selectable as an
optimizer.
* **Tests**
* Added comprehensive tests for orthogonalization, optimizer updates,
bucketing, lr-adjust modes, weight decay, closure behavior, and state
save/load.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->1 parent 9b1df92 commit 82a5f32
File tree
5 files changed
+888
-2
lines changed- deepmd
- pt
- optimizer
- train
- utils
- source/tests/pt
5 files changed
+888
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
2 | 5 | | |
3 | 6 | | |
4 | 7 | | |
5 | 8 | | |
6 | 9 | | |
7 | 10 | | |
8 | 11 | | |
9 | | - | |
| 12 | + | |
0 commit comments