Generating Dataset for Graph Classification

[gdemirezen]

Hello,

I created a dataset for graph classification by creating instances of "torch_geometric.data.Data" and appending them to a list. My list fits into memory. Can I use pyg functions with this dataset structure without implementing a class that inherits from "InMemoryDataset" and therefore, without implementing its necessary methods such as "raw_file_names" or "processed_file_names"?

To use the dataset, I create a data loader using "torch_geometric.loader.DataLoader", I iterate over it and reach node features by data.x and use them in models such as the following. Is this a valid usage of the dataset that is constructed simply as a list of "Data" instances?
e.g.

for data in train_loader:

    data = data.to(device)

    optimizer.zero_grad()  # Clear gradients.

    out = model(data.x, data.edge_index, data.batch, d_o, act)  # Perform a single forward pass.

.....

Additionally, is there a function to perform minmax or standard scalers to this dataset and save the generated scalers for further use? Since I couldn't find this functionality, I created and saved the scaled data before and when performing cross-validation, I load the related dataset file at the beginning of each trial.

Thanks for the great package and for maintaining it so well.

[Padarn]

Yes you can use a torch_geometric.loader.Dataloader for this

dataset = DataLoader(<LIST OF DATA>)
for batch in dataset:
   ...

Perfectly valid.

I don't have an answer for your second question sorry.

[gdemirezen]

Thank you.
So in this case, can we apply PyG functions such as GCNConv, SAGEConv etc. on this dataset by reaching necessary input as batch.x, batch.edge_index and so on?
Actually, can you confirm that we can use a list of Data instances to generate our own graph dataset in PyG instead of creating a Dataset class that inherits from InMemoryDataset?

[rusty1s]

Yes, you can use a list of data objects as well - usage of InMemoryDataset is not required. For normalization, it might be best to iterate over your dataset once and save their dataset statistics. You can then use those to normalize your individual mini-batches.

[gdemirezen]

Thank you for your quick confirmation and guidance on the normalization.

You're right, saving statistics (or calculating them at the beginning) would be better. I see your point. Previously, I thought about scaling the whole dataset but this would require iterating over each data object one by one because we cannot access node features of the whole dataset directly. When we use DataLoader, we can access node features of the whole batch and scale them at once which is very convenient. We don't need to update the dataset list because we don't use it directly as an input to models. I'll implement this method. Thanks!