@@ -249,22 +249,23 @@ def dynamic_lstm(input,
249
249
250
250
h_t & = o_t \odot act_h(c_t)
251
251
252
- where the :math:`W` terms denote weight matrices (e.g. :math:`W_{xi}` is the matrix
253
- of weights from the input gate to the input), :math:`W_{ic}, W_{fc}, W_{oc}`
254
- are diagonal weight matrices for peephole connections. In our implementation,
255
- we use vectors to reprenset these diagonal weight matrices. The :math:`b` terms
256
- denote bias vectors (:math:`b_i` is the input gate bias vector), :math:`\sigma`
257
- is the non-line activations, such as logistic sigmoid function, and
258
- :math:`i, f, o` and :math:`c` are the input gate, forget gate, output gate,
259
- and cell activation vectors, respectively, all of which have the same size as
260
- the cell output activation vector :math:`h`.
261
-
262
- The :math:`\odot` is the element-wise product of the vectors. :math:`act_g` and :math:`act_h`
263
- are the cell input and cell output activation functions and `tanh` is usually
264
- used for them. :math:`\\ tilde{c_t}` is also called candidate hidden state,
265
- which is computed based on the current input and the previous hidden state.
266
-
267
- Set `use_peepholes` False to disable peephole connection. The formula
252
+ where the :math:`W` terms denote weight matrices (e.g. :math:`W_{xi}` is
253
+ the matrix of weights from the input gate to the input), :math:`W_{ic}, \
254
+ W_{fc}, W_{oc}` are diagonal weight matrices for peephole connections. In
255
+ our implementation, we use vectors to reprenset these diagonal weight
256
+ matrices. The :math:`b` terms denote bias vectors (:math:`b_i` is the input
257
+ gate bias vector), :math:`\sigma` is the non-line activations, such as
258
+ logistic sigmoid function, and :math:`i, f, o` and :math:`c` are the input
259
+ gate, forget gate, output gate, and cell activation vectors, respectively,
260
+ all of which have the same size as the cell output activation vector :math:`h`.
261
+
262
+ The :math:`\odot` is the element-wise product of the vectors. :math:`act_g`
263
+ and :math:`act_h` are the cell input and cell output activation functions
264
+ and `tanh` is usually used for them. :math:`\\ tilde{c_t}` is also called
265
+ candidate hidden state, which is computed based on the current input and
266
+ the previous hidden state.
267
+
268
+ Set `use_peepholes` to `False` to disable peephole connection. The formula
268
269
is omitted here, please refer to the paper
269
270
http://www.bioinf.jku.at/publications/older/2604.pdf for details.
270
271
0 commit comments