Merge pull request #784 from yoreG123/fgcnn

Add FGCNN model

Author: frankwhzhang

README_CN.md

@@ -173,6 +173,8 @@ python -u tools/static_trainer.py -m models/rank/dnn/config.yaml #  静态图训
   |   排序   |                  [DSIN](models/rank/dsin/)                                                                          |  -  |       ✓     |     ✓     | >=2.1.0 | [IJCAI 2019][Deep Session Interest Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1905.06482v1.pdf)  |
   |   排序   |                     [SIGN](models/rank/sign/)([文档](https://paddl7erec.readthedocs.io/en/latest/models/rank/sign.html))                                                     |  [Python CPU/GPU](https://aistudio.baidu.com/aistudio/projectdetail/3869111)  |       ✓     |     ✓     | >=2.1.0 | [AAAI 2021][Detecting Beneficial Feature Interactions for Recommender Systems](https://arxiv.org/pdf/2008.00404v6.pdf)                             |
  |   排序   |                                    [IPRec](models/rank/iprec/)([文档](https://paddl7erec.readthedocs.io/en/latest/models/rank/iprec.html))                                    |                                      -                                      |       ✓     |     ✓     | >=2.1.0 | [SIGIR 2021][Package Recommendation with Intra- and Inter-Package Attention Networks](http://nlp.csai.tsinghua.edu.cn/~xrb/publications/SIGIR-21_IPRec.pdf)    |   多任务   |                                                                          [AITM](models/rank/aitm/)                                                                          |  -  |       ✓     |     ✓     | >=2.1.0 | [KDD 2021][Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising](https://arxiv.org/pdf/2105.08489v2.pdf)  |
+   |   排序   |                     [FGCNN](models/rank/fgcnn/)| - |       ✓     |     ✓     | >=2.1.0 | [WWW 2019][Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1904.04447.pdf)                             |
+  |   多任务   |                                                                          [AITM](models/rank/aitm/)                                                                          |  -  |       ✓     |     ✓     | >=2.1.0 | [KDD 2021][Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising](https://arxiv.org/pdf/2105.08489v2.pdf)  |
   |  多任务  |                                  [PLE](models/multitask/ple/)([文档](https://paddlerec.readthedocs.io/en/latest/models/multitask/ple.html))                                   |  [Python CPU/GPU](https://aistudio.baidu.com/aistudio/projectdetail/3238938)  |       ✓     |     ✓     |  >=2.1.0 | [RecSys 2020][Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations](https://dl.acm.org/doi/abs/10.1145/3383313.3412236)                                                              |
   |  多任务  |                                 [ESMM](models/multitask/esmm/)([文档](https://paddlerec.readthedocs.io/en/latest/models/multitask/esmm.html))                                 |  [Python CPU/GPU](https://aistudio.baidu.com/aistudio/projectdetail/3238583)  |       ✓     |     ✓     | >=2.1.0 | [SIGIR 2018][Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://arxiv.org/abs/1804.07931)                                                              |
   |  多任务  |                                 [MMOE](models/multitask/mmoe/)([文档](https://paddlerec.readthedocs.io/en/latest/models/multitask/mmoe.html))                                 |  [Python CPU/GPU](https://aistudio.baidu.com/aistudio/projectdetail/3238934)  |       ✓     |     ✓     | >=2.1.0 | [KDD 2018][Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts](https://dl.acm.org/doi/abs/10.1145/3219819.3220007)                                                       |

README_EN.md

@@ -163,6 +163,7 @@ python -u tools/static_trainer.py -m models/rank/dnn/config.yaml #  Training wit
   |   Rank   |                     [DCN_V2](models/rank/dcn_v2/)                     |  -  |       ✓     |     ✓     | >=2.1.0 | [WWW 2021][DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://arxiv.org/pdf/2008.13535v2.pdf)|
   |   Rank   |                  [DSIN](models/rank/dsin/)                                                                          |  -  |       ✓     |     ✓     | >=2.1.0 | [IJCAI 2019][Deep Session Interest Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1905.06482v1.pdf)  |
   |   Rank   |                     [SIGN](models/rank/sign/)([doc](https://paddlerec.readthedocs.io/en/latest/models/rank/sign.html))                     |  [Python CPU/GPU](https://aistudio.baidu.com/aistudio/projectdetail/3869111)  |       ✓     |     ✓     | >=2.1.0 | [AAAI 2021][Detecting Beneficial Feature Interactions for Recommender Systems](https://arxiv.org/pdf/2008.00404v6.pdf) |
+  |   Rank   |                     [FGCNN](models/rank/fgcnn/)| - |       ✓     |     ✓     | >=2.1.0 | [WWW 2019][Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1904.04447.pdf)                             |
   |   Rank   |                                      [IPRec](models/rank/iprec/)([doc](https://paddl7erec.readthedocs.io/en/latest/models/rank/iprec.html))                                       |                                      -                                      |       ✓     |     ✓     | >=2.1.0 | [SIGIR 2021][Package Recommendation with Intra- and Inter-Package Attention Networks](http://nlp.csai.tsinghua.edu.cn/~xrb/publications/SIGIR-21_IPRec.pdf)                             |
   |   Multi-Task   |                                                                          [AITM](models/rank/aitm/)                                                                          |  -  |       ✓     |     ✓     | >=2.1.0 | [KDD 2021][Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising](https://arxiv.org/pdf/2105.08489v2.pdf)  |
   |      Multi-Task       |                  [PLE](models/multitask/ple/)<br>([doc](https://paddlerec.readthedocs.io/en/latest/models/multitask/ple.html))                   |  [Python CPU/GPU](https://aistudio.baidu.com/aistudio/projectdetail/3238938)  |     ✓     |     ✓     |  >=2.1.0 | [RecSys 2020][Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations](https://dl.acm.org/doi/abs/10.1145/3383313.3412236)                                                              |

contributor.md

@@ -21,5 +21,6 @@
   |                     [MHCN](models/recall/mhcn/)                     |  [Andy1314Chen](https://github.com/Andy1314Chen)  |    https://github.com/PaddlePaddle/PaddleRec/pull/679   | 论文复现赛第五期 |
   |                     [DCN_V2](models/rank/dcn_v2/)                     |  [LinJayan](https://github.com/LinJayan)  |    https://github.com/PaddlePaddle/PaddleRec/pull/677   | 论文复现赛第五期 |
   |                     [SIGN](models/rank/sign/)                     |  [BamLubi](https://github.com/BamLubi)  |    https://github.com/PaddlePaddle/PaddleRec/pull/748   | 论文复现赛第六期 |
+  |                     [FGCNN](models/rank/fgcnn/)                     |  [yoreG123 chenjiyan2001](https://github.com/yoreG123)  |    https://github.com/PaddlePaddle/PaddleRec/pull/784   | 论文复现赛第六期 |
 
 </div> 

datasets/criteo_fgcnn/run.sh

@@ -0,0 +1,7 @@
+wget --no-check-certificate https://paddlerec.bj.bcebos.com/datasets/fgcnn/datapro.zip
+unzip -o datapro.zip	
+echo "Complete data download."
+mkdir train
+mkdir test
+mv criteo_x4_5c863b0f_c15c45a1/train.h5 train
+mv criteo_x4_5c863b0f_c15c45a1/valid.h5 test

doc/source/index.rst

@@ -74,6 +74,7 @@
    models/rank/dlrm.md
    models/rank/dmr.md
    models/rank/dnn.md
+   models/rank/fgcnn.md
    models/rank/ffm.md
    models/rank/fm.md
    models/rank/gatenet.md

doc/source/models/index.rst

@@ -69,6 +69,7 @@ PaddleRec 模型库
    rank/dlrm.md
    rank/dmr.md
    rank/dnn.md
+   rank/fgcnn.md
    rank/ffm.md
    rank/fm.md
    rank/gatenet.md

doc/source/models/rank/fgcnn.md

@@ -0,0 +1,90 @@
+# fgcnn (Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction)
+
+代码请参考：[fgcnn](https://github.com/PaddlePaddle/PaddleRec/tree/master/models/rank/fgcnn)  
+如果我们的代码对您有用，还请点个star啊~  
+
+## 内容
+
+- [模型简介](#模型简介)
+- [数据准备](#数据准备)
+- [运行环境](#运行环境)
+- [快速开始](#快速开始)
+- [效果复现](#效果复现)
+- [进阶使用](#进阶使用)
+- [FAQ](#FAQ)
+
+## 模型简介
+`CTR(Click Through Rate)`，即点击率，是“推荐系统/计算广告”等领域的重要指标，对其进行预估是商品推送/广告投放等决策的基础。本模型实现了下述论文中提出的rank模型：
+
+```text
+@inproceedings{FGCNN,
+  title={Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction},
+  author={Bin Liu, Ruiming Tang, Yingzhi Chen, Jinkai Yu, Huifeng Guo, Yuzhou Zhang},
+  year={2019}
+}
+
+Jieming Zhu, Jinyang Liu, Shuai Yang, Qi Zhang, Xiuqiang He. [Open Benchmarking for Click-Through Rate Prediction](https://arxiv.org/abs/2009.05794). *The 30th ACM International Conference on Information and Knowledge Management (CIKM)*, 2021. [[Bibtex](https://dblp.org/rec/conf/cikm/ZhuLYZH21.html?view=bibtex)]
+
+Jieming Zhu, Kelong Mao, Quanyu Dai, Liangcai Su, Rong Ma, Jinyang Liu, Guohao Cai, Zhicheng Dou, Xi Xiao, Rui Zhang. [BARS: Towards Open Benchmarking for Recommender Systems](https://arxiv.org/pdf/2205.09626.pdf). *The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)*, 2022. [Bibtex]
+```
+
+增加人工的特征通常会提升效果，但是人工设计特征代价很高。因此需要一种自动提取有效特征，丰富特征表示的方式。该工作提出了Feature Generation by Convolutional Neural Network (FGCNN)模型解决该问题。
+FGCNN有两个模块： Feature Generation 和 Deep Classifier。
+其中Feature Generation利用CNN去生成local patterns并且组合生成新的特征。
+Deep Classifier则采用IPNN的结构去学习增强特征空间中的交互。
+该工作表明CTR预测的一个新方向：通过外部的模型减少DNN部分学习高阶特征的难度，本文就是通过CNN+MLP学习的特征，添加到DNN部分。
+
+## 数据准备
+训练及测试数据集选用[Display Advertising Challenge](https://www.kaggle.com/c/criteo-display-ad-challenge/)所用的Criteo数据集。该数据集包括两部分：训练集和测试集。训练集包含一段时间内Criteo的部分流量，测试集则对应训练数据后一天的广告点击流量。
+每一行数据格式如下所示：
+```
+<label> <integer feature 1> ... <integer feature 13> <categorical feature 1> ... <categorical feature 26>
+```
+其中```<label>```表示广告是否被点击，点击用1表示，未点击用0表示。```<integer feature>```代表数值特征（连续特征），共有13个连续特征。```<categorical feature>```代表分类特征（离散特征），共有26个离散特征。相邻两个特征用```\t```分隔，缺失特征用空格表示。测试集中```<label>```特征已被移除。  
+在模型目录的data目录下为您准备了快速运行的示例数据，若需要使用全量数据可以参考下方[效果复现](#效果复现)部分。
+
+## 运行环境
+PaddlePaddle>=2.1
+
+python 3.5/3.6/3.7
+
+os : windows/linux/macos 
+
+## 快速开始
+本文提供了样例数据可以供您快速体验，在fgcnn模型目录的快速执行命令如下： 
+```bash
+# 进入模型目录
+cd models/rank/fgcnn 
+# 动态图训练
+python -u ../../../tools/trainer.py -m config.yaml 
+# 动态图预测
+python -u ../../../tools/infer.py -m config.yaml 
+
+```
+
+## 效果复现
+### 数据集获取及预处理
+为了方便使用者能够快速的跑通每一个模型，我们在每个模型下都提供了样例数据。同时，我们提供了全量数据生成的脚本，将会自动下载转换好格式的criteo数据集。
+在全量数据下模型的指标如下：  
+| 模型 | auc | batch_size | epoch_num| Time of each epoch |
+| :------| :------ | :------ | :------| :------ | 
+| fgcnn |  0.8022   | 2000  |  2  | 约 2 小时 |
+
+1. 确认您当前所在目录为PaddleRec/models/rank/fgcnn
+2. 进入paddlerec/datasets/criteo_fgcnn目录下，执行该脚本，会从国内源的服务器上下载我们预处理完成的criteo全量数据集，并解压到指定文件夹。
+``` bash
+cd ../../../datasets/criteo_fgcnn
+sh run.sh
+``` 
+3. 切回模型目录,执行命令运行全量数据
+```bash
+# 切回模型目录
+cd -
+# 动态图训练
+python -u ../../../tools/trainer.py -m config_bigdata.yaml # 全量数据运行config_bigdata.yaml 
+python -u ../../../tools/infer.py -m config_bigdata.yaml # 全量数据运行config_bigdata.yaml 
+```
+
+## 进阶使用
+  
+## FAQ

doc/source/paddlerec/model_introduce.md

@@ -29,6 +29,7 @@
 ## [din (Deep Interest Network for Click-Through Rate Prediction)](https://github.com/PaddlePaddle/PaddleRec/tree/master/models/rank/din)
 ## [dlrm (Deep Learning Recommendation Model for Personalization and Recommendation Systems)](https://github.com/PaddlePaddle/PaddleRec/tree/master/models/rank/dlrm)
 ## [dmr (Deep Match to Rank Model for Personalized Click-Through Rate Prediction)](https://github.com/PaddlePaddle/PaddleRec/tree/master/models/rank/dmr)
+## [fgcnn (Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction)](https://github.com/PaddlePaddle/PaddleRec/tree/master/models/rank/fgcnn)
 ## [ffm (Field-aware Factorization Machines for CTR Prediction)](https://github.com/PaddlePaddle/PaddleRec/tree/master/models/rank/ffm)
 ## [difm (A Dual Input-aware Factorization Machine for CTR Prediction)](https://github.com/PaddlePaddle/PaddleRec/tree/master/models/rank/difm)
 ## [xdeepfm (xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems)](https://github.com/PaddlePaddle/PaddleRec/tree/master/models/rank/xdeepfm)

doc/source/readme.md

@@ -36,7 +36,8 @@
 [din](https://paddlerec.readthedocs.io/en/latest/models/rank/din.html)  
 [dlrm](https://paddlerec.readthedocs.io/en/latest/models/rank/dlrm.html)  
 [dmr](https://paddlerec.readthedocs.io/en/latest/models/rank/dmr.html)  
-[dnn](https://paddlerec.readthedocs.io/en/latest/models/rank/dnn.html)  
+[dnn](https://paddlerec.readthedocs.io/en/latest/models/rank/dnn.html)
+[fgcnn](https://paddlerec.readthedocs.io/en/latest/models/rank/fgcnn.html)  
 [ffm](https://paddlerec.readthedocs.io/en/latest/models/rank/ffm.html)  
 [fm](https://paddlerec.readthedocs.io/en/latest/models/rank/fm.html)  
 [gatenet](https://paddlerec.readthedocs.io/en/latest/models/rank/gatenet.html)  

models/rank/fgcnn/config.yaml

@@ -0,0 +1,51 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# global settings 
+
+runner:
+  train_data_dir: "data/trainlite"
+  train_reader_path: "reader" # importlib format
+  use_gpu: False
+  use_auc: True
+  train_batch_size: 10
+  epochs: 1
+  print_interval: 10
+  # model_init_path: "output_model_all_fgcnn/1" # init model
+  model_save_path: "output_model_sample_fgcnn"
+  test_data_dir: "data/testlite"
+  infer_reader_path: "reader" # importlib format
+  infer_batch_size: 10
+  infer_load_path: "output_model_sample_fgcnn"
+  infer_start_epoch: 0
+  infer_end_epoch: 1
+
+# hyper parameters of user-defined network
+hyper_parameters:
+  # optimizer config
+  optimizer:
+    class: Adam
+    learning_rate: 0.001
+  sparse_inputs_slots: 26
+  sparse_feature_size: 1000000
+  feature_name: ['I1','I2','I3','I4','I5','I6','I7','I8','I9','I10','I11','I12','I13','C1','C2','C3','C4','C5','C6','C7','C8','C9','C10','C11','C12','C13','C14','C15','C16','C17', 'C18','C19', 'C20', 'C21', 'C22','C23', 'C24', 'C25', 'C26']
+  dense_inputs_slots: 13
+  feature_dim: 5
+  conv_kernel_width: [ 3, 3, 3]
+  conv_filters: [10, 12, 14]
+  new_maps: [3, 3, 3]
+  pooling_width: [2, 2, 2]
+  stride: [1, 1]
+  dnn_hidden_units: [3, 3, 3]
+  dnn_dropout: 0.0