Unsupervised GAT based on unsupervised GraphSAGE

[sciencewu]

Hello! I want to make use of GAT instead of GraphSAGE (graph-sage-unsup-ppi)for my project as I have edge features as well, I will use this architecture and instead of GraphSAGE I defined this simple model:

class GraphAttentionNetworks(torch.nn.Module):
    def __init__(self, 
                in_channels, 
                out_channels,
                edge_dim,
                heads=1,
                p_dropout = 0.4,
                hidden_channels = 16, 
                ):
        super(GraphAttentionNetworks, self).__init__()
        self.p_dropout = p_dropout

        self.conv1 = GATConv(in_channels, 
                            hidden_channels,  
                            edge_dim=edge_dim,      
                            heads=heads
                                    )
        self.conv2 = GATConv(hidden_channels * heads, 
                            out_channels, 
                            edge_dim=edge_dim,
                            heads=heads
                            )

    def forward(self, data):
        # 1. Obtain node embeddings 
        x = F.elu(self.conv1(x=data.x, 
                    edge_index=data.edge_index,
                    edge_attr=data.edge_attr
                    ))
        x = F.dropout(x, p=self.p_dropout, training=self.training)
        x = self.conv2(x=x, 
                    edge_index=data.edge_index,
                    edge_attr=data.edge_attr
                    )
        return x

I train it exactly as it is in the graph-sage-unsup-ppi version. It is a bit unclear how this process learns now, as in GraphSAGE you have the aggregators (mean, max, lstm) that you learn for all the sampling that you do. How would this model aggregate the results over all the subgraphs? It is performing well but I'm not sure if I get what is going on there.
Thanks a lot!

[rusty1s]

GATConv has an aggregator as well (as any GNN layer). Instead of mean, max or LSTM aggregation, GATConv uses attention-based aggregation, that is, it computes attention weights for each neighbor, and then sums all neighbor features up weighted by their attention score.

[sciencewu]

thanks a lot!