Skip to content

Commit 4573a4c

Browse files
renmadarenmada
committed
Add iprec model
2 parents e8667cd + 9c7490c commit 4573a4c

File tree

9 files changed

+109
-94
lines changed

9 files changed

+109
-94
lines changed

‎README_CN.md‎

Lines changed: 63 additions & 63 deletions
Large diffs are not rendered by default.

‎README_EN.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88

99
<h2 align="center">News<img src="./doc/imgs/rec_new_icon.png" width="40"/></h2>
1010

11+
* [2022/5/18] Add 3 algorithms:：[aitm](models/multitask/aitm),[sign](models/rank/sign),[dsin](models/rank/dsin)
1112
* [2022/3/21] Add a new [paper](./paper) directory , show our analysis of the top meeting papers of the recommendation system in 2021 years and the list of recommendation system papers in the industry for your reference.
1213
* [2022/3/10] Add 5 algorithms: [DCN_V2](models/rank/dcn_v2), [MHCN](models/recall/mhcn), [FLEN](models/rank/flen), [Dselect_K](models/multitask/dselect_k)，[AutoFIS](models/rank/autofis)。
1314
* [2022/1/12] Add AI Studio [Online running](https://aistudio.baidu.com/aistudio/projectdetail/3240640) function, you can easily and quickly online experience our model on AI studio platform.
@@ -160,10 +161,10 @@ python -u tools/static_trainer.py -m models/rank/dnn/config.yaml # Training wit
160161
| Rank | [DeepRec](models/rank/deeprec/) | - | ✓ | ✓ | >=2.1.0 | [2017][Training Deep AutoEncoders for Collaborative Filtering](https://arxiv.org/pdf/1708.01715v3.pdf) |
161162
| Rank | [AutoFIS](models/rank/autofis/) | - | ✓ | ✓ | >=2.1.0 | [KDD 2020][AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction](https://arxiv.org/pdf/2003.11235v3.pdf) |
162163
| Rank | [DCN_V2](models/rank/dcn_v2/) | - | ✓ | ✓ | >=2.1.0 | [WWW 2021][DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://arxiv.org/pdf/2008.13535v2.pdf)|
163-
| Rank | [AITM](models/rank/aitm/) | - | ✓ | ✓ | >=2.1.0 | [KDD 2021][Modeling the Sequential Dependence among Audience Multi-step Conversions withMulti-task Learning in Targeted Display Advertising](https://arxiv.org/pdf/2105.08489v2.pdf) |
164164
| Rank | [DSIN](models/rank/dsin/) | - | ✓ | ✓ | >=2.1.0 | [IJCAI 2019][Deep Session Interest Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1905.06482v1.pdf) |
165165
| Rank | [SIGN](models/rank/sign/)([doc](https://paddlerec.readthedocs.io/en/latest/models/rank/sign.html)) | [Python CPU/GPU](https://aistudio.baidu.com/aistudio/projectdetail/3869111) | ✓ | ✓ | >=2.1.0 | [AAAI 2021][Detecting Beneficial Feature Interactions for Recommender Systems](https://arxiv.org/pdf/2008.00404v6.pdf) |
166-
| Rank | [IPRec](models/rank/iprec/)([文档](https://paddl7erec.readthedocs.io/en/latest/models/rank/iprec.html)) | - | ✓ | ✓ | >=2.1.0 | [SIGIR 2021][Package Recommendation with Intra- and Inter-Package Attention Networks](http://nlp.csai.tsinghua.edu.cn/~xrb/publications/SIGIR-21_IPRec.pdf) |
166+
| Rank | [IPRec](models/rank/iprec/)([doc](https://paddl7erec.readthedocs.io/en/latest/models/rank/iprec.html)) | - | ✓ | ✓ | >=2.1.0 | [SIGIR 2021][Package Recommendation with Intra- and Inter-Package Attention Networks](http://nlp.csai.tsinghua.edu.cn/~xrb/publications/SIGIR-21_IPRec.pdf) |
167+
| Multi-Task | [AITM](models/rank/aitm/) | - | ✓ | ✓ | >=2.1.0 | [KDD 2021][Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising](https://arxiv.org/pdf/2105.08489v2.pdf) |
167168
| Multi-Task | [PLE](models/multitask/ple/)<br>([doc](https://paddlerec.readthedocs.io/en/latest/models/multitask/ple.html)) | [Python CPU/GPU](https://aistudio.baidu.com/aistudio/projectdetail/3238938) | ✓ | ✓ | >=2.1.0 | [RecSys 2020][Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations](https://dl.acm.org/doi/abs/10.1145/3383313.3412236) |
168169
| Multi-Task | [ESMM](models/multitask/esmm/)<br>([doc](https://paddlerec.readthedocs.io/en/latest/models/multitask/esmm.html)) | [Python CPU/GPU](https://aistudio.baidu.com/aistudio/projectdetail/3238583) | ✓ | ✓ | >=2.1.0 | [SIGIR 2018][Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://arxiv.org/abs/1804.07931) |
169170
| Multi-Task | [MMOE](models/multitask/mmoe/)<br>([doc](https://paddlerec.readthedocs.io/en/latest/models/multitask/mmoe.html)) | [Python CPU/GPU](https://aistudio.baidu.com/aistudio/projectdetail/3238934) | ✓ | ✓ | >=2.1.0 | [KDD 2018][Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts](https://dl.acm.org/doi/abs/10.1145/3219819.3220007) |

‎models/multitask/mmoe/net.py‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -28,14 +28,15 @@ def __init__(self, feature_size, expert_num, expert_size, tower_size,
2828
self.gate_num = gate_num
2929

3030
self._param_expert = []
31+
expert_init = [pow(10, -i) for i in range(1, self.expert_num + 1)]
3132
for i in range(0, self.expert_num):
3233
linear = self.add_sublayer(
3334
name='expert_' + str(i),
3435
sublayer=nn.Linear(
3536
feature_size,
3637
expert_size,
37-
#initialize the weight randly
38-
weight_attr=nn.initializer.XavierUniform(),
38+
#initialize each expert respectly
39+
weight_attr=nn.initializer.Constant(value=expert_init[i]),
3940
bias_attr=nn.initializer.Constant(value=0.1),
4041
#bias_attr=paddle.ParamAttr(learning_rate=1.0),
4142
name='expert_' + str(i)))
@@ -44,14 +45,15 @@ def __init__(self, feature_size, expert_num, expert_size, tower_size,
4445
self._param_gate = []
4546
self._param_tower = []
4647
self._param_tower_out = []
48+
gate_init = [pow(10, -i) for i in range(1, self.gate_num + 1)]
4749
for i in range(0, self.gate_num):
4850
linear = self.add_sublayer(
4951
name='gate_' + str(i),
5052
sublayer=nn.Linear(
5153
feature_size,
5254
expert_num,
53-
#initialize the weight randly
54-
weight_attr=nn.initializer.XavierUniform(),
55+
#initialize every gate respectly
56+
weight_attr=nn.initializer.Constant(value=gate_init[i]),
5557
bias_attr=nn.initializer.Constant(value=0.1),
5658
#bias_attr=paddle.ParamAttr(learning_rate=1.0),
5759
name='gate_' + str(i)))
@@ -62,8 +64,8 @@ def __init__(self, feature_size, expert_num, expert_size, tower_size,
6264
sublayer=nn.Linear(
6365
expert_size,
6466
tower_size,
65-
#initialize the weight randly
66-
weight_attr=nn.initializer.XavierUniform(),
67+
#initialize each gate respectly
68+
weight_attr=nn.initializer.Constant(value=gate_init[i]),
6769
bias_attr=nn.initializer.Constant(value=0.1),
6870
#bias_attr=paddle.ParamAttr(learning_rate=1.0),
6971
name='tower_' + str(i)))
@@ -74,8 +76,8 @@ def __init__(self, feature_size, expert_num, expert_size, tower_size,
7476
sublayer=nn.Linear(
7577
tower_size,
7678
2,
77-
#initialize the weight randly
78-
weight_attr=nn.initializer.XavierUniform(),
79+
#initialize each gate respectly
80+
weight_attr=nn.initializer.Constant(value=gate_init[i]),
7981
bias_attr=nn.initializer.Constant(value=0.1),
8082
name='tower_out_' + str(i)))
8183
self._param_tower_out.append(linear)

‎models/multitask/ple/net.py‎

Lines changed: 20 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
1+
#the weight randly Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
22
#
33
# Licensed under the Apache License, Version 2.0 (the "License");
44
# you may not use this file except in compliance with the License.
@@ -52,14 +52,15 @@ def __init__(self, feature_size, task_num, exp_per_task, shared_num,
5252
# task tower
5353
self._param_tower = []
5454
self._param_tower_out = []
55+
task_init = [pow(10, -i) for i in range(1, self.task_num + 1)]
5556
for i in range(0, self.task_num):
5657
linear = self.add_sublayer(
5758
name='tower_' + str(i),
5859
sublayer=nn.Linear(
5960
expert_size,
6061
tower_size,
61-
#initialize the weight randly
62-
weight_attr=nn.initializer.XavierUniform(),
62+
#initialize each task respectly
63+
weight_attr=nn.initializer.Constant(value=task_init[i]),
6364
bias_attr=nn.initializer.Constant(value=0.1),
6465
#bias_attr=paddle.ParamAttr(learning_rate=1.0),
6566
name='tower_' + str(i)))
@@ -70,8 +71,8 @@ def __init__(self, feature_size, task_num, exp_per_task, shared_num,
7071
sublayer=nn.Linear(
7172
tower_size,
7273
2,
73-
#initialize the weight randly
74-
weight_attr=nn.initializer.XavierUniform(),
74+
#initialize each task respectly
75+
weight_attr=nn.initializer.Constant(value=task_init[i]),
7576
bias_attr=nn.initializer.Constant(value=0.1),
7677
name='tower_out_' + str(i)))
7778
self._param_tower_out.append(linear)
@@ -113,43 +114,49 @@ def __init__(self, input_feature_size, task_num, exp_per_task, shared_num,
113114

114115
self._param_expert = []
115116
# task-specific expert part
117+
step = self.exp_per_task
116118
for i in range(0, self.task_num):
119+
exp_init = [
120+
pow(10, -k) for k in range(1 + i * step, step * (i + 1) + 1)
121+
]
117122
for j in range(0, self.exp_per_task):
118123
linear = self.add_sublayer(
119124
name=level_name + "_exp_" + str(i) + "_" + str(j),
120125
sublayer=nn.Linear(
121126
input_feature_size,
122127
expert_size,
123-
#initialize the weight randly
124-
weight_attr=nn.initializer.XavierUniform(),
128+
#initialize each expert respectly
129+
weight_attr=nn.initializer.Constant(value=exp_init[j]),
125130
bias_attr=nn.initializer.Constant(value=0.1),
126131
name=level_name + "_exp_" + str(i) + "_" + str(j)))
127132
self._param_expert.append(linear)
128-
133+
shared_exp_init = [pow(10, -i) for i in range(1, self.shared_num + 1)]
129134
# shared expert part
130135
for i in range(0, self.shared_num):
131136
linear = self.add_sublayer(
132137
name=level_name + "_exp_shared_" + str(i),
133138
sublayer=nn.Linear(
134139
input_feature_size,
135140
expert_size,
136-
#initialize the weight randly
137-
weight_attr=nn.initializer.XavierUniform(),
141+
#initialize each shared expert respectly
142+
weight_attr=nn.initializer.Constant(
143+
value=shared_exp_init[i]),
138144
bias_attr=nn.initializer.Constant(value=0.1),
139145
name=level_name + "_exp_shared_" + str(i)))
140146
self._param_expert.append(linear)
141147

142148
# task gate part
143149
self._param_gate = []
144150
cur_expert_num = self.exp_per_task + self.shared_num
151+
gate_init = [pow(10, -i) for i in range(1, self.task_num + 1)]
145152
for i in range(0, self.task_num):
146153
linear = self.add_sublayer(
147154
name=level_name + "_gate_" + str(i),
148155
sublayer=nn.Linear(
149156
input_feature_size,
150157
cur_expert_num,
151-
#initialize the weight randly
152-
weight_attr=nn.initializer.XavierUniform(),
158+
#initialize each gate respectly
159+
weight_attr=nn.initializer.Constant(value=gate_init[i]),
153160
bias_attr=nn.initializer.Constant(value=0.1),
154161
name=level_name + "_gate_" + str(i)))
155162
self._param_gate.append(linear)
@@ -162,8 +169,7 @@ def __init__(self, input_feature_size, task_num, exp_per_task, shared_num,
162169
sublayer=nn.Linear(
163170
input_feature_size,
164171
cur_expert_num,
165-
#initialize the weight randly
166-
weight_attr=nn.initializer.XavierUniform(),
172+
weight_attr=nn.initializer.Constant(value=0.1),
167173
bias_attr=nn.initializer.Constant(value=0.1),
168174
name=level_name + "_gate_shared_"))
169175
self._param_gate_shared = linear

‎models/rank/deeprec/infer.py‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,18 +19,18 @@
1919
import sys
2020
from math import sqrt
2121

22+
__dir__ = os.path.dirname(os.path.abspath(__file__))
23+
print(os.path.abspath('/'.join(__dir__.split('/')[:-3])))
24+
sys.path.append(os.path.abspath(os.path.join(__dir__, '..')))
25+
sys.path.append(os.path.abspath('/'.join(__dir__.split('/')[:-3])))
26+
2227
from tools.utils.utils_single import load_yaml, load_dy_model_class, \
2328
get_abs_model
2429
from tools.utils.save_load import load_model
2530
from paddle.io import DataLoader
2631
import argparse
2732
from importlib import import_module
2833

29-
__dir__ = os.path.dirname(os.path.abspath(__file__))
30-
print(os.path.abspath('/'.join(__dir__.split('/')[:-3])))
31-
sys.path.append(os.path.abspath(os.path.join(__dir__, '..')))
32-
sys.path.append(os.path.abspath('/'.join(__dir__.split('/')[:-3])))
33-
3434
logging.basicConfig(
3535
format='%(asctime)s - %(levelname)s - %(message)s', level=logging.INFO)
3636
logger = logging.getLogger(__name__)

‎models/rank/fat_deepffm/config.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ runner:
2626
print_interval: 10
2727

2828
model_save_path: "output_model_fat_deepffm"
29-
infer_batch_size: 1000
29+
infer_batch_size: 1
3030
infer_reader_path: "criteo_reader" # importlib format
3131
test_data_dir: "data/sample_data/train"
3232

‎models/rank/sign/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@ pip install pgl
127127
cd - # 切回模型目录
128128
# 动态图训练
129129
python -u ../../../tools/trainer.py -m config_bigdata.yaml # 全量数据运行
130-
python -u .././../tools/infer.py -m config_bigdata.yaml # 全量数据预测
130+
python -u ../../../tools/infer.py -m config_bigdata.yaml # 全量数据预测
131131
```
132132

133133
## 进阶使用

‎tools/infer.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ def main(args):
6565

6666
# tools.vars
6767
use_gpu = config.get("runner.use_gpu", True)
68+
use_auc = config.get("runner.use_auc", False)
6869
use_xpu = config.get("runner.use_xpu", False)
6970
use_npu = config.get("runner.use_npu", False)
7071
use_visual = config.get("runner.use_visual", False)
@@ -177,6 +178,8 @@ def main(args):
177178
metric_str += (
178179
metric_list_name[metric_id] +
179180
": {:.6f},".format(metric_list[metric_id].accumulate()))
181+
if use_auc:
182+
metric_list[metric_id].reset()
180183

181184
tensor_print_str = ""
182185
if tensor_print_dict is not None:

‎tools/trainer.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ def main(args):
6565

6666
# tools.vars
6767
use_gpu = config.get("runner.use_gpu", True)
68+
use_auc = config.get("runner.use_auc", False)
6869
use_npu = config.get("runner.use_npu", False)
6970
use_xpu = config.get("runner.use_xpu", False)
7071
use_visual = config.get("runner.use_visual", False)
@@ -191,6 +192,8 @@ def main(args):
191192
metric_str += (
192193
metric_list_name[metric_id] +
193194
": {:.6f},".format(metric_list[metric_id].accumulate()))
195+
if use_auc:
196+
metric_list[metric_id].reset()
194197

195198
tensor_print_str = ""
196199
if tensor_print_dict is not None:

0 commit comments

Comments
 (0)