Skip to content

Commit 82a5f32

Browse files
authored
feat(pt): add AdaMuon optimizer (#5130)
<!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added AdaMuon optimizer for PyTorch training with mixed 2D/1D parameter handling, batched orthogonalized updates, momentum and per-parameter state. * **Configuration** * New optimizer options: momentum, weight decay, Adam betas, lr adjustment modes and tuning coefficients; AdaMuon selectable as an optimizer. * **Tests** * Added comprehensive tests for orthogonalization, optimizer updates, bucketing, lr-adjust modes, weight decay, closure behavior, and state save/load. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 9b1df92 commit 82a5f32

File tree

5 files changed

+888
-2
lines changed

5 files changed

+888
-2
lines changed

deepmd/pt/optimizer/__init__.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,12 @@
11
# SPDX-License-Identifier: LGPL-3.0-or-later
2+
from .adamuon import (
3+
AdaMuonOptimizer,
4+
)
25
from .KFWrapper import (
36
KFOptimizerWrapper,
47
)
58
from .LKF import (
69
LKFOptimizer,
710
)
811

9-
__all__ = ["KFOptimizerWrapper", "LKFOptimizer"]
12+
__all__ = ["AdaMuonOptimizer", "KFOptimizerWrapper", "LKFOptimizer"]

0 commit comments

Comments
 (0)