How to use different dataloader for different GPU in DDP

[xiachenrui]

Hi, all

I really like lightning! Now, I want to train a GNN on some big graphs (200+ graphs, each graph contains 1M nodes), it is hard to load them into memory once and use DDP. I want to know if it is possible to allocate each GPU process with a Dataloader.

I am also hesitating whether to use IterableDataset, it seems hard to combine IterableDataset and DDP traning.

Any other suggestions will be useful !

Best,
Chen-Rui