Create a TU(like)Dataset on disk

[Devanshu24]

Hi!
Is there a simple way to create an on-disk dataset for a dataset of the form of TUDataset, i.e. if I have sufficient memory I can simply use read_tu_data to create an InMemoryDataset however there isn't as much memory as is needed. I understand that PyG does provide a way to write your custom process method, however, I was wondering given that it's already in the form of TUDataset, can we somehow use it, and not write a complete process method for an on-disk dataset?
It is completely understandable if there isn't a way, just wanted to check in once in case if there is a way.

Thank you!

[rusty1s]

This is an interesting question. Currently, read_tu_data will read the complete dataset in memory.

However, I think this is possible nonetheless. Given that you know about the number of nodes and the number of edges of each graph in your dataset, you should be able to read node features and graph-structure in chunks via pandas.read_csv.

[Devanshu24]

Thanks! I did think that we won't be able to use read_tu_data since it makes everything into single lists at places.
Thanks for your help :)