On the future of pyg-lib

[flandolfi]

Hi @rusty1s,

I'm following pyg-lib since it was an empty repo, and now I see it's growing very fast! :)

I was wandering, will pyg-lib eventually merge pytorch-scatter, -cluster, -sparse in a single, multipurpose repo (maybe with substituting old matmul wit NVIDIA CUTLASS?

I'm getting to the point: I'm about to work on re-implmementing and definine new pooling methods that would be most useful if implemented using the C++/CUDA API. Exemples are:

• "Revisiting Edge Pooling in Graph Neural Networks", by Landolfi;
• "Generalizing Downsampling from Regular Data to Graphs", by Bacciu, Conte, e Landolfi;

aswell as all renowed hierarchical spectral clustering methods laike Graclus (the real one!), METIS, or hierarchical modularity ones, like Louvain and/or Leiden (as in cuGraph).

My qestion is: is pyg-lib ripe enough to starting working wih it? Most of my methods are built using high-level python calls in PyG, but what do they really need are just gather/scatter operations. I tried building those by using the C++ API of torch-scatter, but I was good only to make it "standalone" (merging the CMakeLists of both scatter, cluster, sparse was a bit of an hell.

Can you guide me on this? If you think this could be too early or too onerous, just say that, and I'll tick to python calls :)

Sincerely,
Francesco

[rusty1s]

Hi @flandolfi,

the current long-term goal is to deprecate torch-scatter, torch-sparse and torch-cluster, and either upstream these operators to PyTorch (e.g., graph-agnostic operators) or integrate them in pyg-lib (e.g., graph-specific operators). We have already started with this effort, e.g, via torch.scatter_reduce or torch.spmm. Here is a slide that presents this idea more clearly:

I am happy to integrate the methods you talk about in pyg-lib, but if these methods are only built using high-level python calls, why not stay in Python directly?

[flandolfi]

Hi, thank you for your fastest reply!

First, I'm happy to see that the "unification" is happening, this should simplify coding a LOT.

I am happy to integrate the methods you talk about in pyg-lib, but if these methods are only built using high-level python calls, why not stay in Python directly?

My idea was indeed developing clustering/covering algorithms that given a featured graph returned just a vector or a (sparse) cover matrix. (This was more or less like the matching algorithm used in Graclus that is already present in torch-cluster.)

Yes, I could use just pythorch calls (all my algorithms are JIT-able), but If there is this possibility, I'd like to refactor the code so that the algorithm is already at its "optimum". Moreover, hierarchical algorithm like GraClus and Metis may 1) have a common structure cluster-reduce-refine that it may suite better for C++ API 2) operate multiple times for single training iteration and should have efficiency at highest priority.

Anyway, I may put first the pure JITable python code for the clustering, and then start working on the CUDA/C++ implementation in pyg-lib, if it sounds good to you!

Thank you a lot,
Francesco

[rusty1s]

This definitely sounds good to me, keep me posted.

[Xurorx]

Hello @rusty1s , based on above discussion, I wonder if PyG team has deprecated torch-scatter, torch-sparse and torch-cluster and integrate them in pyg-lib2.0?

[wsad1]

@Xurorx from pyg2.3 onwards torch-scatter , torch-sparse etc have become optional. https://github.com/pyg-team/pytorch_geometric/releases/tag/2.3.0.

[Xurorx]

Thank you!