Skip to content

Commit 91dc7c6

Browse files
shenweichenshenweichen
authored
update EDCN &fix typo
update EDCN &fix typo
1 parent c13aba6 commit 91dc7c6

File tree

23 files changed

+309
-218
lines changed

23 files changed

+309
-218
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 33 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
name: CI
1+
name: CI_TF2
22

3-
on:
3+
on:
44
push:
55
path:
66
- 'deepctr/*'
@@ -9,17 +9,17 @@ on:
99
path:
1010
- 'deepctr/*'
1111
- 'tests/*'
12-
12+
1313
jobs:
1414
build:
1515

1616
runs-on: ubuntu-latest
1717
timeout-minutes: 180
1818
strategy:
1919
matrix:
20-
python-version: [3.6,3.7,3.8,3.9,3.10.7]
21-
tf-version: [1.4.0,1.15.0,2.6.0,2.7.0,2.8.0,2.9.0,2.10.0]
22-
20+
python-version: [ 3.6,3.7,3.8, 3.9,3.10.7 ]
21+
tf-version: [ 2.6.0,2.7.0,2.8.0,2.9.0,2.10.0 ]
22+
2323
exclude:
2424
- python-version: 3.7
2525
tf-version: 1.4.0
@@ -64,31 +64,31 @@ jobs:
6464
- python-version: 3.10.7
6565
tf-version: 2.7.0
6666
steps:
67-
68-
- uses: actions/checkout@v3
69-
70-
- name: Setup python environment
71-
uses: actions/setup-python@v4
72-
with:
73-
python-version: ${{ matrix.python-version }}
7467

75-
- name: Install dependencies
76-
run: |
77-
pip3 install -q tensorflow==${{ matrix.tf-version }}
78-
pip install -q protobuf==3.19.0
79-
pip install -q requests
80-
pip install -e .
81-
- name: Test with pytest
82-
timeout-minutes: 180
83-
run: |
84-
pip install -q pytest
85-
pip install -q pytest-cov
86-
pip install -q python-coveralls
87-
pytest --cov=deepctr --cov-report=xml
88-
- name: Upload coverage to Codecov
89-
uses: codecov/[email protected]
90-
with:
91-
token: ${{secrets.CODECOV_TOKEN}}
92-
file: ./coverage.xml
93-
flags: pytest
94-
name: py${{ matrix.python-version }}-tf${{ matrix.tf-version }}
68+
- uses: actions/checkout@v3
69+
70+
- name: Setup python environment
71+
uses: actions/setup-python@v4
72+
with:
73+
python-version: ${{ matrix.python-version }}
74+
75+
- name: Install dependencies
76+
run: |
77+
pip3 install -q tensorflow==${{ matrix.tf-version }}
78+
pip install -q protobuf==3.19.0
79+
pip install -q requests
80+
pip install -e .
81+
- name: Test with pytest
82+
timeout-minutes: 180
83+
run: |
84+
pip install -q pytest
85+
pip install -q pytest-cov
86+
pip install -q python-coveralls
87+
pytest --cov=deepctr --cov-report=xml
88+
- name: Upload coverage to Codecov
89+
uses: codecov/[email protected]
90+
with:
91+
token: ${{secrets.CODECOV_TOKEN}}
92+
file: ./coverage.xml
93+
flags: pytest
94+
name: py${{ matrix.python-version }}-tf${{ matrix.tf-version }}

‎.github/workflows/ci2.yml‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
name: CI_TF1
2+
3+
on:
4+
push:
5+
path:
6+
- 'deepctr/*'
7+
- 'tests/*'
8+
pull_request:
9+
path:
10+
- 'deepctr/*'
11+
- 'tests/*'
12+
13+
jobs:
14+
build:
15+
16+
runs-on: ubuntu-latest
17+
timeout-minutes: 180
18+
strategy:
19+
matrix:
20+
python-version: [ 3.6,3.7 ]
21+
tf-version: [ 1.15.0 ]
22+
23+
exclude:
24+
- python-version: 3.7
25+
tf-version: 1.4.0
26+
- python-version: 3.7
27+
tf-version: 1.12.0
28+
- python-version: 3.7
29+
tf-version: 1.15.0
30+
- python-version: 3.8
31+
tf-version: 1.4.0
32+
- python-version: 3.8
33+
tf-version: 1.14.0
34+
- python-version: 3.8
35+
tf-version: 1.15.0
36+
- python-version: 3.6
37+
tf-version: 2.7.0
38+
- python-version: 3.6
39+
tf-version: 2.8.0
40+
- python-version: 3.6
41+
tf-version: 2.9.0
42+
- python-version: 3.6
43+
tf-version: 2.10.0
44+
- python-version: 3.9
45+
tf-version: 1.4.0
46+
- python-version: 3.9
47+
tf-version: 1.15.0
48+
- python-version: 3.9
49+
tf-version: 2.2.0
50+
- python-version: 3.9
51+
tf-version: 2.5.0
52+
- python-version: 3.9
53+
tf-version: 2.6.0
54+
- python-version: 3.9
55+
tf-version: 2.7.0
56+
- python-version: 3.10.7
57+
tf-version: 1.4.0
58+
- python-version: 3.10.7
59+
tf-version: 1.15.0
60+
- python-version: 3.10.7
61+
tf-version: 2.2.0
62+
- python-version: 3.10.7
63+
tf-version: 2.5.0
64+
- python-version: 3.10.7
65+
tf-version: 2.6.0
66+
- python-version: 3.10.7
67+
tf-version: 2.7.0
68+
steps:
69+
70+
- uses: actions/checkout@v3
71+
72+
- name: Setup python environment
73+
uses: actions/setup-python@v4
74+
with:
75+
python-version: ${{ matrix.python-version }}
76+
77+
- name: Install dependencies
78+
run: |
79+
pip3 install -q tensorflow==${{ matrix.tf-version }}
80+
pip install -q protobuf==3.19.0
81+
pip install -q requests
82+
pip install -e .
83+
- name: Test with pytest
84+
timeout-minutes: 180
85+
run: |
86+
pip install -q pytest
87+
pip install -q pytest-cov
88+
pip install -q python-coveralls
89+
pytest --cov=deepctr --cov-report=xml
90+
- name: Upload coverage to Codecov
91+
uses: codecov/[email protected]
92+
with:
93+
token: ${{secrets.CODECOV_TOKEN}}
94+
file: ./coverage.xml
95+
flags: pytest
96+
name: py${{ matrix.python-version }}-tf${{ matrix.tf-version }}

‎deepctr/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
from .utils import check_version
22

3-
__version__ = '0.9.2'
3+
__version__ = '0.9.3'
44
check_version(__version__)

‎deepctr/layers/__init__.py‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,16 @@
11
import tensorflow as tf
22

33
from .activation import Dice
4-
from .core import DNN, LocalActivationUnit, PredictionLayer, RegulationLayer
4+
from .core import DNN, LocalActivationUnit, PredictionLayer, RegulationModule
55
from .interaction import (CIN, FM, AFMLayer, BiInteractionPooling, CrossNet, CrossNetMix,
66
InnerProductLayer, InteractingLayer,
77
OutterProductLayer, FGCNNLayer, SENETLayer, BilinearInteraction,
8-
FieldWiseBiInteraction, FwFMLayer, FEFMLayer, BridgeLayer)
8+
FieldWiseBiInteraction, FwFMLayer, FEFMLayer, BridgeModule)
99
from .normalization import LayerNormalization
1010
from .sequence import (AttentionSequencePoolingLayer, BiasEncoding, BiLSTM,
1111
KMaxPooling, SequencePoolingLayer, WeightedSequenceLayer,
12-
Transformer, DynamicGRU,PositionEncoding)
13-
14-
from .utils import NoMask, Hash, Linear, _Add, combined_dnn_input, softmax, reduce_sum
12+
Transformer, DynamicGRU, PositionEncoding)
13+
from .utils import NoMask, Hash, Linear, _Add, combined_dnn_input, softmax, reduce_sum, Concat
1514

1615
custom_objects = {'tf': tf,
1716
'InnerProductLayer': InnerProductLayer,
@@ -28,7 +27,6 @@
2827
'SequencePoolingLayer': SequencePoolingLayer,
2928
'AttentionSequencePoolingLayer': AttentionSequencePoolingLayer,
3029
'CIN': CIN,
31-
'RegulationLayer': RegulationLayer,
3230
'InteractingLayer': InteractingLayer,
3331
'LayerNormalization': LayerNormalization,
3432
'BiLSTM': BiLSTM,
@@ -39,6 +37,7 @@
3937
'FGCNNLayer': FGCNNLayer,
4038
'Hash': Hash,
4139
'Linear': Linear,
40+
'Concat': Concat,
4241
'DynamicGRU': DynamicGRU,
4342
'SENETLayer': SENETLayer,
4443
'BilinearInteraction': BilinearInteraction,
@@ -50,5 +49,6 @@
5049
'FEFMLayer': FEFMLayer,
5150
'reduce_sum': reduce_sum,
5251
'PositionEncoding': PositionEncoding,
53-
'BridgeLayer': BridgeLayer
52+
'RegulationModule': RegulationModule,
53+
'BridgeModule': BridgeModule
5454
}

‎deepctr/layers/core.py‎

Lines changed: 12 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -267,41 +267,39 @@ def get_config(self, ):
267267
return dict(list(base_config.items()) + list(config.items()))
268268

269269

270-
class RegulationLayer(Layer):
270+
class RegulationModule(Layer):
271271
"""Regulation module used in EDCN.
272272
273273
Input shape
274-
- A list of 3D tensor with shape: ``(batch_size,1,embedding_size)``.
274+
- 3D tensor with shape: ``(batch_size,field_size,embedding_size)``.
275275
276276
Output shape
277-
- 2D tensor with shape: ``(batch_size, embedding_size * field_num)``.
277+
- 2D tensor with shape: ``(batch_size,field_size * embedding_size)``.
278278
279279
Arguments
280280
- **tau** : Positive float, the temperature coefficient to control
281281
distribution of field-wise gating unit.
282282
283-
- **seed** : A Python integer to use as random seed.
284-
285283
References
286284
- [Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models.](https://dlp-kdd.github.io/assets/pdf/DLP-KDD_2021_paper_12.pdf)
287285
"""
288286

289-
def __init__(self, tau=0.1, **kwargs):
287+
def __init__(self, tau=1.0, **kwargs):
290288
if tau == 0:
291-
raise ValueError("RegulationLayer tau can not be zero.")
289+
raise ValueError("RegulationModule tau can not be zero.")
292290
self.tau = 1.0 / tau
293-
super(RegulationLayer, self).__init__(**kwargs)
291+
super(RegulationModule, self).__init__(**kwargs)
294292

295293
def build(self, input_shape):
296-
self.field_num = int(input_shape[1])
294+
self.field_size = int(input_shape[1])
297295
self.embedding_size = int(input_shape[2])
298296
self.g = self.add_weight(
299-
shape=(1, self.field_num, 1),
297+
shape=(1, self.field_size, 1),
300298
initializer=Ones(),
301299
name=self.name + '_field_weight')
302300

303301
# Be sure to call this somewhere!
304-
super(RegulationLayer, self).build(input_shape)
302+
super(RegulationModule, self).build(input_shape)
305303

306304
def call(self, inputs, **kwargs):
307305

@@ -311,13 +309,13 @@ def call(self, inputs, **kwargs):
311309

312310
feild_gating_score = tf.nn.softmax(self.g * self.tau, 1)
313311
E = inputs * feild_gating_score
314-
return tf.reshape(E, [-1, self.field_num * self.embedding_size])
312+
return tf.reshape(E, [-1, self.field_size * self.embedding_size])
315313

316314
def compute_output_shape(self, input_shape):
317-
return (None, self.field_num * self.embedding_size)
315+
return (None, self.field_size * self.embedding_size)
318316

319317
def get_config(self):
320318
config = {'tau': self.tau}
321-
base_config = super(RegulationLayer, self).get_config()
319+
base_config = super(RegulationModule, self).get_config()
322320
base_config.update(config)
323321
return base_config

‎deepctr/layers/interaction.py‎

Lines changed: 19 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1493,48 +1493,45 @@ def get_config(self):
14931493
return config
14941494

14951495

1496-
class BridgeLayer(Layer): # ridge
1497-
"""AttentionPoolingLayer layer used in EDCN
1496+
class BridgeModule(Layer):
1497+
"""Bridge Module used in EDCN
14981498
14991499
Input shape
1500-
- A list of 3D tensor with shape: ``(batch_size,1,embedding_size)``. Its length is ``number of subnetworks``.
1500+
- A list of two 2D tensor with shape: ``(batch_size, units)``.
15011501
15021502
Output shape
1503-
- 2D tensor with shape: ``(batch_size, embedding_size)``.
1503+
- 2D tensor with shape: ``(batch_size, units)``.
15041504
15051505
Arguments
1506-
- **activation**: Activation function to use.
1507-
1508-
- **l2_reg**: float between 0 and 1. L2 regularizer strength applied to the kernel weights matrix.
1506+
- **bridge_type**: The type of bridge interaction, one of 'pointwise_addition', 'hadamard_product', 'concatenation', 'attention_pooling'
15091507
1510-
- **seed**: A Python integer to use as random seed.
1508+
- **activation**: Activation function to use.
15111509
15121510
References
15131511
- [Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models.](https://dlp-kdd.github.io/assets/pdf/DLP-KDD_2021_paper_12.pdf)
15141512
15151513
"""
15161514

1517-
def __init__(self, bridge_type='attention_pooling', activation='relu', l2_reg=0, seed=1024, **kwargs):
1515+
def __init__(self, bridge_type='hadamard_product', activation='relu', **kwargs):
15181516
self.bridge_type = bridge_type
15191517
self.activation = activation
1520-
self.l2_reg = l2_reg
1521-
self.seed = seed
15221518

1523-
super(BridgeLayer, self).__init__(**kwargs)
1519+
super(BridgeModule, self).__init__(**kwargs)
15241520

15251521
def build(self, input_shape):
15261522
if not isinstance(input_shape, list) or len(input_shape) < 2:
15271523
raise ValueError(
1528-
'A `AttentionPoolingLayer` layer should be called '
1529-
'on a list of at least 2 inputs')
1524+
'A `BridgeModule` layer should be called '
1525+
'on a list of 2 inputs')
15301526

15311527
self.dnn_dim = int(input_shape[0][-1])
1528+
if self.bridge_type == "concatenation":
1529+
self.dense = Dense(self.dnn_dim, self.activation)
1530+
elif self.bridge_type == "attention_pooling":
1531+
self.dense_x = DNN([self.dnn_dim, self.dnn_dim], self.activation, output_activation='softmax')
1532+
self.dense_h = DNN([self.dnn_dim, self.dnn_dim], self.activation, output_activation='softmax')
15321533

1533-
self.dense = Dense(self.dnn_dim, self.activation)
1534-
self.dense_x = DNN([self.dnn_dim, self.dnn_dim], output_activation='softmax')
1535-
self.dense_h = DNN([self.dnn_dim, self.dnn_dim], output_activation='softmax')
1536-
1537-
super(BridgeLayer, self).build(input_shape) # Be sure to call this somewhere!
1534+
super(BridgeModule, self).build(input_shape) # Be sure to call this somewhere!
15381535

15391536
def call(self, inputs, **kwargs):
15401537
x, h = inputs
@@ -1543,7 +1540,7 @@ def call(self, inputs, **kwargs):
15431540
elif self.bridge_type == "hadamard_product":
15441541
return x * h
15451542
elif self.bridge_type == "concatenation":
1546-
return self.dense(tf.concat(inputs, axis=-1))
1543+
return self.dense(tf.concat([x, h], axis=-1))
15471544
elif self.bridge_type == "attention_pooling":
15481545
a_x = self.dense_x(x)
15491546
a_h = self.dense_h(h)
@@ -1553,12 +1550,10 @@ def compute_output_shape(self, input_shape):
15531550
return (None, self.dnn_dim)
15541551

15551552
def get_config(self):
1556-
base_config = super(BridgeLayer, self).get_config().copy()
1553+
base_config = super(BridgeModule, self).get_config().copy()
15571554
config = {
15581555
'bridge_type': self.bridge_type,
1559-
'l2_reg': self.l2_reg,
1560-
'activation': self.activation,
1561-
'seed': self.seed
1556+
'activation': self.activation
15621557
}
15631558
config.update(base_config)
15641559
return config

0 commit comments

Comments
 (0)