Skip to content

Commit 8b79c57

Browse files
author
cossio
committed
add comment
1 parent a34cc85 commit 8b79c57

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/optimise/optimisers.jl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -545,7 +545,7 @@ function apply!(o::AdaBelief, x, Δ)
545545
The original implementation (https://github.com/juntang-zhuang/Adabelief-Optimizer)
546546
uses the square of Adam's epsilon, which we do here.
547547
See also: https://github.com/juntang-zhuang/Adabelief-Optimizer/issues/61 =#
548-
eps2 = o.epsilon^2
548+
eps2 = o.epsilon^2 # TODO: make epsilon^2 the default in next breaking release
549549

550550
@. mt = β[1] * mt + (1 - β[1]) * Δ
551551
@. st = β[2] * st + (1 - β[2]) *- mt) * conj- mt) + eps2

0 commit comments

Comments
 (0)