Adding "Efficient Graph Convolution" (EGC)

[shyam196]

Efficient Graph Convolutions reference code / models

We've just released this work building on the previous work here in Cambridge on PNA. It is a drop-in replacement for the likes of GCN/GAT/GIN/MPNN/PNA/etc with low memory usage but still -- surprisingly -- SOTA performance. Given the relative lack of tuning required + the generality of the method, I think it'd be useful to add to PyG.

Please feel free to have a play with our code / pretrained models by the way. We'd love to hear how it works for you.

There are two aspects discussed in the paper:

1. The layer implementation -- I am more than happy to clean up the reference version and upstream it.
2. Aggregator Fusion -- the idea here is that if you are doing any kind of sparse operations, you can perform multiple types of aggregators "for free" since you spend so much time idling waiting for data from memory. This will likely require major changes to both torch_sparse / torch_scatter to incorporate, but it will also be useful for methods like PNA. This bit doesn't have to be added immediately but I think it's definitely useful going forward.

Happy to contribute this myself if you're happy! 🙂

[rusty1s]

Thanks for starting this conversation. I'm well aware of your work and the results are awesome :) I actually already had plans to implement it by myself, but I would really appreciate it if you are willing to contribute, i.e., the layer implementation and a simple example on ogbg-molhiv.

Regarding aggregator fusion, I definitely agree, but we should probably incorporate that later on. I'm thinking of a simple interface like:

scatter(..., reduce=['mean, 'max', 'min'])

or

adj_t.matmul(x, reduce=['mean, 'max', 'min'])

This would be really cool to integrate :)

[shyam196]

Awesome! Will take a look at this for you over the coming week.

Totally agreed re: interface for the scatter/spmm. And also agree that it's lower priority.

[rusty1s]

As far as I see, aggregator fusion currently only works in inference mode, as there is no backward pass implemented. Is that true?

[shyam196]

It can only be applied on the forward pass I think. I think you will be able to use it on the backward pass in some cases (e.g. if all your aggregators are additive) but I don't believe this applies in general -- e.g. if you're using the sum and max aggregators at the same time, the backwards pass for each looks totally different.

The code in the repo also fuses the weighting step into the kernel, but this is definitely not required in general. So you would speedup the forward pass, but not backwards.