Merge pull request #998 from tensorlayer/nlp

NLP: Seq2seq, LuongAttention, SimpleRNN, GRURNN, LSTMRNN

Author: JingqingZ

CHANGELOG.md

@@ -91,6 +91,9 @@ To release a new version, please update the changelog as followed:
 - Add version_info in model.config. (PR #992)
 - Replace tf.nn.func with tf.nn.func.\_\_name\_\_ in model config.
 - Add Reinforcement learning tutorials. (PR #995)
+- Add RNN layers with simple rnn cell, GRU cell, LSTM cell. (PR #998)
+- Update Seq2seq (#998) 
+- Add Seq2seqLuongAttention model (#998)
 
 ### Fixed
 
@@ -100,20 +103,21 @@ To release a new version, please update the changelog as followed:
 - @Tokarev-TT-33: #995
 - @initial-h: #995
 - @Officium: #995
+- @ArnoldLIULJ: #998
+- @JingqingZ: #998
+
 
 ## [2.0.2] - 2019-6-5
 
 ### Changed
 - change the format of network config, change related code and files; change layer act (PR #980)
-- update Seq2seq (#989) 
 
 ### Fixed
 - Fix dynamic model cannot track PRelu weights gradients problem (PR #982)
 - Raise .weights warning (commit)
 
 ### Contributors
 - @warshallrho: #980
-- @ArnoldLIULJ: #989
 - @1FengL: #982
 
 ## [2.0.1] - 2019-5-17

docs/modules/layers.rst

@@ -80,11 +80,15 @@ Layer list
    SwitchNorm
 
    RNN
+   SimpleRNN
+   GRURNN
+   LSTMRNN
    BiRNN
 
    retrieve_seq_length_op
    retrieve_seq_length_op2
    retrieve_seq_length_op3
+   target_mask_op
 
    Flatten
    Reshape
@@ -579,6 +583,18 @@ RNN layer
 """"""""""""""""""""""""""
 .. autoclass:: RNN
 
+RNN layer with Simple RNN Cell
+""""""""""""""""""""""""""""""""""
+.. autoclass:: SimpleRNN
+
+RNN layer with GRU Cell
+""""""""""""""""""""""""""""""""""
+.. autoclass:: GRURNN
+
+RNN layer with LSTM Cell
+""""""""""""""""""""""""""""""""""
+.. autoclass:: LSTMRNN
+
 Bidirectional layer
 """""""""""""""""""""""""""""""""
 .. autoclass:: BiRNN
@@ -593,13 +609,16 @@ Compute Sequence length 1
 .. autofunction:: retrieve_seq_length_op
 
 Compute Sequence length 2
-""""""""""""""""""""""""""
+"""""""""""""""""""""""""""""
 .. autofunction:: retrieve_seq_length_op2
 
 Compute Sequence length 3
-""""""""""""""""""""""""""
+""""""""""""""""""""""""""""
 .. autofunction:: retrieve_seq_length_op3
 
+Compute mask of the target sequence
+"""""""""""""""""""""""""""""""""""""""
+.. autofunction:: target_mask_op
 
 
 

docs/modules/models.rst

@@ -13,6 +13,9 @@ TensorLayer provides many pretrained models, you can easily use the whole or a p
     VGG19
     SqueezeNetV1
     MobileNetV1
+    Seq2seq
+    Seq2seqLuongAttention
+
 
 Base Model
 -----------
@@ -37,3 +40,14 @@ MobileNetV1
 ----------------
 
 .. autofunction:: MobileNetV1
+
+Seq2seq
+------------------------
+
+.. autoclass:: Seq2seq
+
+
+Seq2seq Luong Attention
+------------------------
+
+.. autoclass:: Seq2seqLuongAttention