Support for Distributed Full-batch Training on large graphs

[zhuangbility111]

Hi! Does PyG provide the solution for the distributed full-batch training on large-scale graphs (ex. obgn-papers100M)?

[rusty1s]

Distributed data parallel does not work in full-batch mode. What you can theoretically do is to use distributed training with NeighborLoader with batch_size=num_nodes // num_processes, which would be equivalent to full-batch training.