GAE training with node-level data splitting to generalize to graph extensions

[dpaysan]

I want to apply a GAE to downstream perform node clustering. However, to assess the performance of the GAE I do not yet want to look at the clustering of the node embeddings but solely evaluate the performance using the reconstruction loss of the adjacency matrix.

I am planning to split my node set randomly into train, validation and test set.
The idea would be the following:

• During training: use the subnetwork induced by the training node set for message passing, loss computation and backpropagation
• During validation: use the subnetwork induced by the union of the training and validation node sets for message passing and the subnework induced by solely the validation nodes to compute the loss
• During testing: use the complete network for message passing and the subnetwork induced by solely the test nodes to compute the loss

The idea would be to train an autoencoder that can also generalize to unseen nodes potentially later added to the graph structure.

Is that a reasonable approach? I am happy for any kind of feedback.

[rusty1s]

That sounds actually quite similar to what we do in RandomLinkSplit and in our examples/autoencoder.py example (except that we actually try to infer new links in the sense that we perform evaluation and message passing on different sets of links). In your case, this does not seem to be necessary, if I understand you correctly.

[dpaysan]

It is indeed - the rational behind that approach is actually motivated by the example. In contrast to the RandomLinkSpit we would however ensure that all links associated to a certain node set (train, val and test) are masked in the corresponding data splits. Similar to the link prediction task I thought it would make sense to perform evaluation and message passing on different sets of links in that setting as I cannot think of a reason why that wouldnt be appropriate in the given setup. Am I missing something or would you agree?

On that note is there an existing way to get all negative_edge_indices i.e. all true negative edges? In contrast to the code in example/autoencoder.py I would like to not sample those negative edges when the evaluating the model during validation and testing to not introduce further uncertainty in those scores and avoid ensuring that the negative edges are sampled in such a way that they correspond to non-existent edges between any combination of nodes in the respective validation or test set.

Thanks a lot for your help!

[rusty1s]

Sounds great. For obtaining all negative edges, you can do something like:

neg_adj = torch.ones(num_nodes, num_nodes, dtype=torch.bool)
neg_adj[edge_index[0], edge_index[1]] = False
neg_edge_index = neg_adj.nonzero(as_tuple=False).t()

Keep in mind that this may consume a lot of memory, given that edge_index is usually sparse.

[dpaysan]

Thanks! Yep, the size of the network especially after subsetting them to the validation or test nodes are relatively small such that memory shouldnt be an issue.