Would it make sense to initialize weights for GATConv using "Kaiming He" instead of "glorot"?

[jgwak1]

Hi,

I am thinking of apply "kaiming He" weight-initialization to GATConv layer weights.
( As I am planning to use the ReLU activation function )

However, it appears GATConv weight-initialization is internally set as "Xavier (glorot")".

My question is, would there be a reason to not replace the "glorot" with "kaiming He" weight-intialization?

        # In case we are operating in bipartite graphs, we apply separate
        # transformations 'lin_src' and 'lin_dst' to source and target nodes:
        if isinstance(in_channels, int):
            self.lin_src = Linear(in_channels, heads * out_channels,
                                  bias=False, weight_initializer='glorot')
            self.lin_dst = self.lin_src
        else:
            self.lin_src = Linear(in_channels[0], heads * out_channels, False,
                                  weight_initializer='glorot')
            self.lin_dst = Linear(in_channels[1], heads * out_channels, False,
                                  weight_initializer='glorot')

        # The learnable parameters to compute attention coefficients:
        self.att_src = Parameter(torch.Tensor(1, heads, out_channels))
        self.att_dst = Parameter(torch.Tensor(1, heads, out_channels))

        if edge_dim is not None:
            self.lin_edge = Linear(edge_dim, heads * out_channels, bias=False,
                                   weight_initializer='glorot')
            self.att_edge = Parameter(torch.Tensor(1, heads, out_channels))
        else:
            self.lin_edge = None
            self.register_parameter('att_edge', None)

[rusty1s]

AFAIK, glorot is used in the official implementation, so I would likely not want to change the default. You can either override the initialization manually, or we could think about providing different strategies for initialization as part of the input arguments of GATConv. Happy to take a PR on this one.