Skip to content

Commit 7f5c0b1

Browse files
committed
fix
1 parent 2efb2cc commit 7f5c0b1

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

_posts/2025-11-25-d2l_optimization.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -405,7 +405,13 @@ Adadelta 是 AdaGrad 的变体,减少学习率对坐标的适应性
405405

406406
- $\mathbf{s}_t = \rho \mathbf{s}_{t-1} + (1 - \rho) \mathbf{g}_t^2$($\rho$ 为超参数)
407407

408-
- 调整梯度:$\mathbf{g}_t' = \frac{\sqrt{\Delta\mathbf{x}_{t-1} + \epsilon}}{\sqrt{{\mathbf{s}_t + \epsilon}}} \odot \mathbf{g}_t$($\epsilon$ 为小值,如 1e-5,保证数值稳定)
408+
- 调整梯度:
409+
410+
$$
411+
\mathbf{g}_t' = \frac{\sqrt{\Delta\mathbf{x}_{t-1} + \epsilon}}{\sqrt{{\mathbf{s}_t + \epsilon}}} \odot \mathbf{g}_t
412+
$$
413+
414+
$\epsilon$ 为小值,如 1e-5,保证数值稳定)
409415

410416
- 参数更新:$\mathbf{x}_t = \mathbf{x}_{t-1} - \mathbf{g}_t'$
411417

0 commit comments

Comments
 (0)