How to create the Dataloader

[chenyuqi990215]

Hi,

I'm pretty new to the field of graph neural networks and pytorch geometric.

Each data point in my data consists of a graph with N point and E edges (e.g. features with dimension NC and edge_index with dimension E2), together with an input sequence (e.g. dimension 1*D). How could I create such a dataset and enable mini-batch sampling.

Is it possible to solve the problem with rewriting the collate_fn while create DataLoader?

(I would like to use GCNConv in torch_geometry, therefore I would like for each batch sample, features with dimension (B*N)C, edge_indexs with dimension (BE)2 and sequences with dimension BN)

Thank you in advance.

[rusty1s]

Saving your data as

data = Data(x=x, edge_index=edge_index, seq=seq)

where

x = ... [N, C]
edge_index = ... [2, E]
seq = [1, D]

should already give you a Batch object with shapes:

x = ... [B * N, C]
edge_index = ... [2, B * E]
seq = [B, D]

which can directly be used as input to GCNConv. Hope this helps :)

[chenyuqi990215]

Pretty thank for your answer. I come up with another question. Since the graph is static (i.e. the graph is the same for all the data point), and also the graph is every large (i.e. 60000+ edges). Is there some better ways of data preparing?

[rusty1s]

A current workaround is to save data without edge_index, and instead save it as a separate file in process and load it in __init__:

self.edge_index = torch.load(path_to_edge_index)

You can then override the get() method as follows:

def get(self, idx):
    data = super().get(idx)
    data.edge_index = self.edge_index
    return data