Missing node types on some graphs leads to errors in GCN

[carlosSorcero]

Hello,

I am reaching out to the community to seek guidance on how to approach an issue I am facing. I am getting None on some keys (node types) in the forward function of my custom GCN. I believe this may be happening because some of the graphs may not have the said node type at all. Is this something we have to guard against in our models? Are there any recommended ways to either guard against it or to make the layers invariable to it?

Here is my GCN:

class HAN(torch.nn.Module):
    def __init__(self, in_channels,
                 out_channels, hidden_channels=128, heads=8, metadata=None, num_classes=1):
        super().__init__()
        self.metadata = metadata
        self.hidden_channels = hidden_channels
        self.han_conv = HANConv(in_channels, self.hidden_channels, heads=heads,
                                dropout=dropout, metadata=metadata)
        self.lin1 = Linear(self.hidden_channels * len(metadata[0]), out_channels)
        self.lin2 = Linear(out_channels, num_classes)

    def forward(self, x_dict, edge_index_dict, batch):
        x = self.han_conv(x_dict, edge_index_dict)

        concat = []
        for node_type in self.metadata[0]:
            concat.append(global_mean_pool(x[node_type], batch[node_type]))
        x = torch.cat(concat, dim=-1)
        
        # Apply a final classifier
        x = F.dropout(x, p=dropout, training=self.training)
        x = self.lin1(x).relu()
        x = torch.sigmoid(self.lin2(x))

        x = torch.reshape(x, (-1,))

        return x

If I print the x_dict, I indeed see a key with None as value. For example, on a collection of graphs generated by FakeHeteroDataset:

{'v0': None, 'v1': tensor([[1.3987, 0.4811, 0.0000,  ..., 1.0205, 0.0000, 0.5383],
        [0.4290, 0.0852, 0.0000,  ..., 0.0000, 0.0000, 0.9854],
        [0.6086, 0.0000, 0.2825,  ..., 0.4621, 0.0000, 0.6149],
        ...,
        [0.0000, 0.0000, 0.4693,  ..., 0.3314, 0.2921, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.5381, 1.2595, 0.0000],
        [0.0000, 0.4780, 0.5386,  ..., 0.5918, 0.2786, 0.1338]],
       device='cuda:0', grad_fn=<SumBackward1>), 'v2': tensor([[0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.2963, 0.0000,  ..., 1.1078, 0.0000, 0.0198],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.3071, 0.2022],
        ...,
        [0.5840, 0.0000, 0.0852,  ..., 0.0000, 0.9572, 0.0433],
        [0.0000, 0.0540, 0.0000,  ..., 0.0000, 0.6579, 0.5732],
        [0.0000, 0.0197, 0.0000,  ..., 0.0000, 0.0000, 0.1095]],
       device='cuda:0', grad_fn=<SumBackward1>)}

Any help would be appreciated. Thanks!

Happy holidays!

[rusty1s]

In general, this does happen if there exists a node type that is not included as a destination node type in any edge type, i.e., there exists no edge pointing to this particular node type. Can you confirm that this is not the case?

[carlosSorcero]

In other words: do we have to guard against missing connections in the GCN? If so, what is the recommended way to do so? I’m mostly interested in graph classification and my graphs are likely to be small and potentially missing connections to some node types, thus this is something I need to consider. Off the cuff ideas:

• making sure all graphs have at least one connection to all node types (maybe a self loop).
• check for None in the GCN and dynamically average values of other node types / edge types
• Check for None in GCN and add a 0 vector or similar.

I’m not particularly happy with either, but maybe you have other suggestions?

[rusty1s]

I think the best solution is to have a random edge type that points to this particular node type with no edges, i.e., edge_index has shape [2, 0]. We can potentially integrate this functionality internally into HeteroConv and to_hetero.

[carlosSorcero]

I’ll try that, thanks! Would be great if the convolution layers would do this check internally too.

[rusty1s]

Yes, let me look into that :)

[rusty1s]

I added a check into to_hetero and HeteroConv to capture this unintended usage, see #3775. I'm not sure yet whether we really want to auto-correct this internally, as this is clearly a miss-usage of the heterogeneous GNN modules. In the end, it doesn't make a lot of sense to have node types that do not occur as destination nodes types in any edge type.