-
Notifications
You must be signed in to change notification settings - Fork 78
Closed
Milestone
Description
Summary networks in general are still poorly tested. Upon introducing more rigorous testing, I encountered some issues.
Set Transformer
@stefanradev93
The set transformer (along with its child classes like MAB, PMA, etc.) appears to have multiple issues:
- It cannot build automatically, the user has to supply the input dimension.
- Unlike other networks, it does not properly handle converting user-friendly inputs like
dropout=Noneordropout=0. - Its argument naming conventions differ from the other networks, e.g.
num_attention_blocksinstead ofdepth. - It relies on manually named argument forwarding, e.g.
num_dense_feedforwardcannot easily be renamed without sweeping the whole directory.
How do you suggest we move forward with this? Upon writing more rigorous tests, I find (1) particularly bothersome.
(to be expanded)
Metadata
Metadata
Assignees
Labels
No labels