How GATConv does attention in HeteroData?

[Nokimann]

In Heterogeneous Graph Learning,
GATConv was introduced as an example for Creating Heterogeneous GNNs

from torch_geometric.nn import GATConv, Linear, to_hetero

class GAT(torch.nn.Module):
    def __init__(self, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = GATConv((-1, -1), hidden_channels, add_self_loops=False)
        self.lin1 = Linear(-1, hidden_channels)
        self.conv2 = GATConv((-1, -1), out_channels, add_self_loops=False)
        self.lin2 = Linear(-1, out_channels)

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index) + self.lin1(x)
        x = x.relu()
        x = self.conv2(x, edge_index) + self.lin2(x)
        return x

model = GAT(hidden_channels=64, out_channels=dataset.num_classes)
model = to_hetero(model, data.metadata(), aggr='sum')

Let's say

type1_n1 - > target
type1_n2 - > target
type2_n1 - > target
type2_n2 - > target

The attention may perform on type1 and type2 separately and then aggregate.
Or, the attention may perform on type1 and type2 together (project the type1 and type2 on the same space to coexist in the target space and then do attention) and then aggregate.
I think the to_hetero and HeteroConv convert like the former.

[rusty1s]

Yes, attention is computed indepedently for every edge type, i.e., attention is performed on type1 and type2 individually. If you want to combine the computation of attention for both edge types, it's probably best to merge them into a single edge type.

[DogitoErgoSum]

I have one question about this. If my graph has only one node per node type, it is useless to use attention because accordingly to this equation:

The attention coefficients will always be 1 because i have only one neighbor per node type at max . Is this correct?
@rusty1s

[rusty1s]

I think this is correct for graphs where every node has a degree not greater than 1. However, keep in mind that attention is still useful in case you have a single node of a node type that is connected to multiple nodes of another node type.