Add multitask models：SharedBottom, ESSM, MMOE, PLE

- add multitask learning models：SharedBottom, ESSM, MMOE, PLE
- set `dnn_hidden_units` parameter to `(256, 128, 64)`
- add new package `deepctr.models.multitask` and `deepctr.models.sequence` 
- fix truncated feature value error when padding string sequence in run_multivalue_movielens_hash.py

Author: shenweichen

.github/ISSUE_TEMPLATE/bug_report.md

@@ -20,7 +20,7 @@ Steps to reproduce the behavior:
 **Operating environment(运行环境):**
  - python version [e.g. 3.6, 3.7]
  - tensorflow version [e.g. 1.4.0, 1.15.0, 2.5.0]
- - deepctr version [e.g. 0.8.6,]
+ - deepctr version [e.g. 0.9.0,]
 
 **Additional context**
 Add any other context about the problem here.

.github/ISSUE_TEMPLATE/question.md

@@ -17,4 +17,4 @@ Add any other context about the problem here.
 **Operating environment(运行环境):**
  - python version [e.g. 3.6]
  - tensorflow version [e.g. 1.4.0, 1.15.0, 2.5.0]
- - deepctr version [e.g. 0.8.6,]
+ - deepctr version [e.g. 0.9.0,]

.github/workflows/ci.yml

@@ -18,7 +18,7 @@ jobs:
     strategy:
       matrix:
         python-version: [3.6,3.7]
-        tf-version: [1.4.0,1.15.0,2.1.0,2.5.0]
+        tf-version: [1.4.0,1.15.0,2.2.0,2.5.0]
 
         exclude:
           - python-version: 3.7
@@ -28,10 +28,10 @@ jobs:
 
     steps:
 
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v2
 
     - name: Setup python environment
-      uses: actions/setup-python@v1
+      uses: actions/setup-python@v2.2.2
       with:
         python-version: ${{ matrix.python-version }}
 
@@ -49,7 +49,7 @@ jobs:
         pip install -q python-coveralls
         pytest --cov=deepctr --cov-report=xml
     - name: Upload coverage to Codecov  
-      uses: codecov/codecov-action@v1.0.2
+      uses: codecov/codecov-action@v2.0.3
       with:
         token: ${{secrets.CODECOV_TOKEN}}
         file: ./coverage.xml

README.md

@@ -18,18 +18,23 @@
 <!-- [![Gitter](https://badges.gitter.im/DeepCTR/community.svg)](https://gitter.im/DeepCTR/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) -->
 
 
-DeepCTR is a **Easy-to-use**,**Modular** and **Extendible** package of deep-learning based CTR models along with lots of core components layers which can be used to easily build custom models.You can use any complex model with `model.fit()`，and `model.predict()` .
-
-- Provide `tf.keras.Model` like interface for **quick experiment**. [example](https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html#getting-started-4-steps-to-deepctr)
-- Provide  `tensorflow estimator` interface for **large scale data** and **distributed training**. [example](https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html#getting-started-4-steps-to-deepctr-estimator-with-tfrecord)
+DeepCTR is a **Easy-to-use**,**Modular** and **Extendible** package of deep-learning based CTR models along with lots of
+core components layers which can be used to easily build custom models.You can use any complex model with `model.fit()`
+，and `model.predict()` .
+
+- Provide `tf.keras.Model` like interface for **quick experiment**
+  . [example](https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html#getting-started-4-steps-to-deepctr)
+- Provide  `tensorflow estimator` interface for **large scale data** and **distributed training**
+  . [example](https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html#getting-started-4-steps-to-deepctr-estimator-with-tfrecord)
 - It is compatible with both `tf 1.x`  and `tf 2.x`.
 
 Some related projects:
+
 - DeepMatch: https://github.com/shenweichen/DeepMatch
 - DeepCTR-Torch: https://github.com/shenweichen/DeepCTR-Torch
 
-
-Let's [**Get Started!**](https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html)([Chinese Introduction](https://zhuanlan.zhihu.com/p/53231955)) and [welcome to join us!](./CONTRIBUTING.md)
+Let's [**Get Started!**](https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html)([Chinese
+Introduction](https://zhuanlan.zhihu.com/p/53231955)) and [welcome to join us!](./CONTRIBUTING.md)
 
 ## Models List
 
@@ -45,8 +50,8 @@ Let's [**Get Started!**](https://deepctr-doc.readthedocs.io/en/latest/Quick-Star
 |   Attentional Factorization Machine    | [IJCAI 2017][Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](http://www.ijcai.org/proceedings/2017/435) |
 |      Neural Factorization Machine      | [SIGIR 2017][Neural Factorization Machines for Sparse Predictive Analytics](https://arxiv.org/pdf/1708.05027.pdf)                                               |
 |                xDeepFM                 | [KDD 2018][xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://arxiv.org/pdf/1803.05170.pdf)                         |
-|         Deep Interest Network          | [KDD 2018][Deep Interest Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1706.06978.pdf)     
-|                AutoInt                 | [CIKM 2019][AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://arxiv.org/abs/1810.11921)                              ||
+|         Deep Interest Network          | [KDD 2018][Deep Interest Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1706.06978.pdf)     |
+|                AutoInt                 | [CIKM 2019][AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://arxiv.org/abs/1810.11921)                              |
 |    Deep Interest Evolution Network     | [AAAI 2019][Deep Interest Evolution Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1809.03672.pdf)                                            |
 |                FwFM                    | [WWW 2018][Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising](https://arxiv.org/pdf/1806.03514.pdf)                |
 |                  ONN                  | [arxiv 2019][Operation-aware Neural Networks for User Response Prediction](https://arxiv.org/pdf/1904.12579.pdf)                                                |
@@ -59,11 +64,15 @@ Let's [**Get Started!**](https://deepctr-doc.readthedocs.io/en/latest/Quick-Star
 |                DCN V2                    | [arxiv 2020][DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://arxiv.org/abs/2008.13535)   |
 |                DIFM                 | [IJCAI 2020][A Dual Input-aware Factorization Machine for CTR Prediction](https://www.ijcai.org/Proceedings/2020/0434.pdf)   |
 |   FEFM and DeepFEFM                    | [arxiv 2020][Field-Embedded Factorization Machines for Click-through rate prediction](https://arxiv.org/abs/2009.09931)                                         |
+|              SharedBottom               | [arxiv 2017][An Overview of Multi-Task Learning in Deep Neural Networks](https://arxiv.org/pdf/1706.05098.pdf)  |
+|   ESMM                    | [SIGIR 2018][Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://arxiv.org/abs/1804.07931)                       |
+|   MMOE                    | [KDD 2018][Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts](https://dl.acm.org/doi/abs/10.1145/3219819.3220007)                   |
+|   PLE                    | [RecSys 2020][Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations](https://dl.acm.org/doi/10.1145/3383313.3412236)                   |
 
 ## Citation
 
-- Weichen Shen. (2017). DeepCTR: Easy-to-use,Modular and Extendible package of deep-learning based CTR models. https://github.com/shenweichen/deepctr.
-
+- Weichen Shen. (2017). DeepCTR: Easy-to-use,Modular and Extendible package of deep-learning based CTR
+  models. https://github.com/shenweichen/deepctr.
 
 If you find this code useful in your research, please cite it using the following BibTeX:
 
@@ -81,11 +90,10 @@ If you find this code useful in your research, please cite it using the followin
 ## DisscussionGroup
 
 - [Discussions](https://github.com/shenweichen/DeepCTR/discussions)
-- 公众号：**浅梦学习笔记**  
-- wechat ID: **deepctrbot**
+- 公众号：**浅梦学习笔记**
+- wechat ID: **deepctrbot**
 
   ![wechat](./docs/pics/code.png)
-  
 
 ## Main contributors([welcome to join us!](./CONTRIBUTING.md))
 
@@ -108,6 +116,11 @@ If you find this code useful in your research, please cite it using the followin
         ​ <a href="https://github.com/pandeconscious">Harshit Pande</a>
         <p> Amazon   </p>​
       </td>
+      <td>
+        ​ <a href="https://github.com/morningsky"><img width="70" height="70" src="https://github.com/morningsky.png?s=40" alt="pic"></a><br>
+        ​ <a href="https://github.com/morningsky">Lai Mincai</a>
+        <p> ShanghaiTech University </p>​
+      </td>
       <td>
         ​ <a href="https://github.com/codewithzichao"><img width="70" height="70" src="https://github.com/codewithzichao.png?s=40" alt="pic"></a><br>
         ​ <a href="https://github.com/codewithzichao">Li Zichao</a>

deepctr/__init__.py

@@ -1,4 +1,4 @@
 from .utils import check_version
 
-__version__ = '0.8.7'
+__version__ = '0.9.0'
 check_version(__version__)

deepctr/estimator/feature_column.py

@@ -47,6 +47,6 @@ def input_from_feature_columns(features, feature_columns, l2_reg_embedding=0.0):
 def is_embedding(feature_column):
     try:
         from tensorflow.python.feature_column.feature_column_v2 import EmbeddingColumn
-    except:
+    except ImportError:
         EmbeddingColumn = _EmbeddingColumn
     return isinstance(feature_column, (_EmbeddingColumn, EmbeddingColumn))

deepctr/estimator/models/autoint.py

@@ -20,7 +20,7 @@
 
 def AutoIntEstimator(linear_feature_columns, dnn_feature_columns, att_layer_num=3, att_embedding_size=8, att_head_num=2,
                      att_res=True,
-                     dnn_hidden_units=(256, 256), dnn_activation='relu', l2_reg_linear=1e-5,
+                     dnn_hidden_units=(256, 128, 64), dnn_activation='relu', l2_reg_linear=1e-5,
                      l2_reg_embedding=1e-5, l2_reg_dnn=0, dnn_use_bn=False, dnn_dropout=0, seed=1024,
                      task='binary', model_dir=None, config=None, linear_optimizer='Ftrl',
                      dnn_optimizer='Adagrad', training_chief_hooks=None):

deepctr/estimator/models/ccpm.py

@@ -19,7 +19,7 @@
 
 
 def CCPMEstimator(linear_feature_columns, dnn_feature_columns, conv_kernel_width=(6, 5), conv_filters=(4, 4),
-                  dnn_hidden_units=(256,), l2_reg_linear=1e-5, l2_reg_embedding=1e-5, l2_reg_dnn=0, dnn_dropout=0,
+                  dnn_hidden_units=(128, 64), l2_reg_linear=1e-5, l2_reg_embedding=1e-5, l2_reg_dnn=0, dnn_dropout=0,
                   seed=1024, task='binary', model_dir=None, config=None, linear_optimizer='Ftrl',
                   dnn_optimizer='Adagrad', training_chief_hooks=None):
     """Instantiates the Convolutional Click Prediction Model architecture.

deepctr/estimator/models/dcn.py

@@ -15,7 +15,7 @@
 from ...layers.utils import combined_dnn_input
 
 
-def DCNEstimator(linear_feature_columns, dnn_feature_columns, cross_num=2, dnn_hidden_units=(128, 128,),
+def DCNEstimator(linear_feature_columns, dnn_feature_columns, cross_num=2, dnn_hidden_units=(256, 128, 64),
                  l2_reg_linear=1e-5,
                  l2_reg_embedding=1e-5,
                  l2_reg_cross=1e-5, l2_reg_dnn=0, seed=1024, dnn_dropout=0, dnn_use_bn=False,

deepctr/estimator/models/deepfefm.py

@@ -19,7 +19,7 @@
 
 
 def DeepFEFMEstimator(linear_feature_columns, dnn_feature_columns,
-                      dnn_hidden_units=(128, 128), l2_reg_linear=0.00001, l2_reg_embedding_feat=0.00001,
+                      dnn_hidden_units=(256, 128, 64), l2_reg_linear=0.00001, l2_reg_embedding_feat=0.00001,
                       l2_reg_embedding_field=0.00001, l2_reg_dnn=0, seed=1024, dnn_dropout=0.0,
                       dnn_activation='relu', dnn_use_bn=False, task='binary', model_dir=None,
                       config=None, linear_optimizer='Ftrl', dnn_optimizer='Adagrad', training_chief_hooks=None):