Skip to content

Commit 3b0eff6

Browse files
author
Yibing Liu
committed
Format the writing in doc of dynamic_lstm
1 parent f46257f commit 3b0eff6

File tree

1 file changed

+17
-16
lines changed
  • python/paddle/v2/fluid/layers

1 file changed

+17
-16
lines changed

python/paddle/v2/fluid/layers/nn.py

Lines changed: 17 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -249,22 +249,23 @@ def dynamic_lstm(input,
249249
250250
h_t & = o_t \odot act_h(c_t)
251251
252-
where the :math:`W` terms denote weight matrices (e.g. :math:`W_{xi}` is the matrix
253-
of weights from the input gate to the input), :math:`W_{ic}, W_{fc}, W_{oc}`
254-
are diagonal weight matrices for peephole connections. In our implementation,
255-
we use vectors to reprenset these diagonal weight matrices. The :math:`b` terms
256-
denote bias vectors (:math:`b_i` is the input gate bias vector), :math:`\sigma`
257-
is the non-line activations, such as logistic sigmoid function, and
258-
:math:`i, f, o` and :math:`c` are the input gate, forget gate, output gate,
259-
and cell activation vectors, respectively, all of which have the same size as
260-
the cell output activation vector :math:`h`.
261-
262-
The :math:`\odot` is the element-wise product of the vectors. :math:`act_g` and :math:`act_h`
263-
are the cell input and cell output activation functions and `tanh` is usually
264-
used for them. :math:`\\tilde{c_t}` is also called candidate hidden state,
265-
which is computed based on the current input and the previous hidden state.
266-
267-
Set `use_peepholes` False to disable peephole connection. The formula
252+
where the :math:`W` terms denote weight matrices (e.g. :math:`W_{xi}` is
253+
the matrix of weights from the input gate to the input), :math:`W_{ic}, \
254+
W_{fc}, W_{oc}` are diagonal weight matrices for peephole connections. In
255+
our implementation, we use vectors to reprenset these diagonal weight
256+
matrices. The :math:`b` terms denote bias vectors (:math:`b_i` is the input
257+
gate bias vector), :math:`\sigma` is the non-line activations, such as
258+
logistic sigmoid function, and :math:`i, f, o` and :math:`c` are the input
259+
gate, forget gate, output gate, and cell activation vectors, respectively,
260+
all of which have the same size as the cell output activation vector :math:`h`.
261+
262+
The :math:`\odot` is the element-wise product of the vectors. :math:`act_g`
263+
and :math:`act_h` are the cell input and cell output activation functions
264+
and `tanh` is usually used for them. :math:`\\tilde{c_t}` is also called
265+
candidate hidden state, which is computed based on the current input and
266+
the previous hidden state.
267+
268+
Set `use_peepholes` to `False` to disable peephole connection. The formula
268269
is omitted here, please refer to the paper
269270
http://www.bioinf.jku.at/publications/older/2604.pdf for details.
270271

0 commit comments

Comments
 (0)