Skip to content

Commit c57d52d

Browse files
authored
Make logσ(x) = -softplus(-x)
See also discussion in issue #102
1 parent 93e4fee commit c57d52d

File tree

1 file changed

+7
-11
lines changed

1 file changed

+7
-11
lines changed

src/activation.jl

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -23,19 +23,15 @@ end
2323
2424
Return `log(σ(x))` which is computed in a numerically stable way.
2525
26-
julia> logσ(0.)
26+
julia> logσ(0)
2727
-0.6931471805599453
28-
julia> logσ.([-100, -10, 100.])
28+
julia> logσ.([-100, -10, 100])
2929
3-element Array{Float64,1}:
30-
-100.0
31-
-10.0
32-
-0.0
33-
"""
34-
function logσ(x)
35-
max_v = max(zero(x), -x)
36-
z = exp(-max_v) + exp(-x-max_v)
37-
return -(max_v + log(z))
38-
end
30+
-100.0
31+
-10.000045398899218
32+
-3.720075976020836e-44
33+
"""
34+
logσ(x) = -softplus(-x)
3935
const logsigmoid = logσ
4036

4137

0 commit comments

Comments
 (0)