Skip to content

SetTransformer and other Summary Network Issues #207

@LarsKue

Description

@LarsKue

Summary networks in general are still poorly tested. Upon introducing more rigorous testing, I encountered some issues.

Set Transformer

@stefanradev93
The set transformer (along with its child classes like MAB, PMA, etc.) appears to have multiple issues:

  1. It cannot build automatically, the user has to supply the input dimension.
  2. Unlike other networks, it does not properly handle converting user-friendly inputs like dropout=None or dropout=0.
  3. Its argument naming conventions differ from the other networks, e.g. num_attention_blocks instead of depth.
  4. It relies on manually named argument forwarding, e.g. num_dense_feedforward cannot easily be renamed without sweeping the whole directory.

How do you suggest we move forward with this? Upon writing more rigorous tests, I find (1) particularly bothersome.

(to be expanded)

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions