Skip to content

Commit f224029

Browse files
author
ranqiu
committed
Refine multi_head_attention
1 parent d29901b commit f224029

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

python/paddle/trainer_config_helpers/networks.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1586,9 +1586,9 @@ def multi_head_attention(query,
15861586
value_proj, offset=value_proj_size * i, size=value_proj_size)
15871587

15881588
if attention_type == 'dot-product attention':
1589-
m = linear_comb_layer(
1590-
weights=sub_query_proj,
1591-
vectors=sub_key_proj,
1589+
m = dot_prod_layer(
1590+
input1=sub_query_proj,
1591+
input2=sub_key_proj,
15921592
name='%s_dot-product_%d' % (name, i))
15931593
m = slope_intercept_layer(
15941594
input=m,

0 commit comments

Comments
 (0)