Autoencoder on neighbourhoods with batching

[azchrinn]

Hello!

Thanks for all the hard work on making this package available to the community. I’ve been lurking around here for the past years, and i really like the direction of the project. Keep it up!

Now, onto my question.

I am trying to build a graph autoencoder to operate on neighbourhoods of nodes. I have some number of larger graphs that i would like to sample these neighbourhoods from, encode them, and reconstruct their adjacency and/or features on this per-neighbourhood basis. So far i think i have gotten this to work with batch sizes of 1.

In this particular case i use the

NeighborLoader(graph,
                num_neighbors=[-1, -1, -1],
                batch_size=1,
                shuffle=True)

to sample my 3 hop neighbourhoods (or subgraphs) one at a time for use with the GAE or VGAE implementation. One NeighborLoader for each of my larger graphs, iterating over them to draw some number of samples (not exhaustively). Currently i consider having iterated over all the loaders 1 epoch during the training of the autoencoder model.

This works reasonably well it seems, but i would like to extend this to batch sizes larger than 1. I’ve seen some similar issues on this topic, but i have to confess i do not fully follow how one should go about doing the extension.

In particular, i have some trouble understanding how i would alter the negative sampling in the (V)GAE recon loss to not accidentally create negative edges between the disconnected neighbourhoods of the batch.

I suppose from one perspective, i would want there to be negative edges between these nodes such that the model does not try to make positive edges there? On the other hand though, during the sampling i cannot be sure that the neighbourhoods actually are disconnected.
Maybe by chance the neighbourhoods will be sampled close enough together that some of their nodes actually overlap and there thus should be a connection between some nodes?

I would much appreciate any ideas.

Thanks!