Skip to content
This repository was archived by the owner on Jul 10, 2025. It is now read-only.

Commit 6b1b201

Browse files
authored
Update 20200616-keras-multihead-attention.md
Add mark to authors. Add plan for addons migration.
1 parent 0472095 commit 6b1b201

File tree

1 file changed

+10
-1
lines changed

1 file changed

+10
-1
lines changed

rfcs/20200616-keras-multihead-attention.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
| Status | (Proposed / Accepted / Implemented / Obsolete) |
44
| :------------ | :------------------------------------------------------ |
55
| **RFC #** | [260](https://github.com/tensorflow/community/pull/260) |
6-
| **Author(s)** | Hongkun Yu ([email protected]) |
6+
| **Author(s)** | Hongkun Yu ([email protected]), Mark Omernick ([email protected]) |
77
| **Sponsor** | Francois Chollet ([email protected]) |
88
| **Updated** | 2020-06-16 |
99

@@ -242,6 +242,15 @@ we would like to introduce an optional argument `attention_mask` for
242242
the shape is (batch_size, target_length, source_length). Whenever
243243
`attention_mask` is specified, the `mask` argument is OK to be skipped.
244244

245+
* TFA `MultiHeadAttention` Deprecation and Re-mapping
246+
247+
[MultiHeadAttention](https://github.com/tensorflow/addons/blob/master/tensorflow_addons/layers/multihead_attention.py) has been released. The proposed `MultiHeadAttention` has similar `__init__` arguments
248+
and `call` interface, where the minor differences are argument names and the attention `mask` shape.
249+
We expect the new `MultiHeadAttention` keras layer will
250+
cover the functionalities. Once the implementation are merged as experimental layers,
251+
we will work with TF Addons team to design the deprecation and re-mapping procedure.
252+
253+
245254
### Alternatives Considered
246255

247256
We examined multi-head attention layer implemented in various libraries. There

0 commit comments

Comments
 (0)