Best way to store graphs

[DogitoErgoSum]

Hi
I need to create around 2.4M individual graphs.
Right now I m using the Dataset class to create and store each graph. This approach is rather slow when loading a batch of graphs using dataloader and each graph takes around 28MB to store on disk.
I checked that if I instead stored a batch of graphs using Batch.from_data_list([data(),data1(),...]), the whole batch takes way less memory to store than the sum of each individual graph. However, i cannot use dataloader to create a batch of batches.
Is there a way around this?

[fengsxy]

So do you mean that you want store the data for one by one ,so that you can import it ome by one.That will increase the spead of it?
If so,i tackle my data one by one ,and write a merge function.

[rusty1s]

Yes, this approach should speed up things by a lot. Last time I checked creating batches out of a list a Batch objects worked just fine. Alternatively, you may want to convert your loaded Batch into a list of Data objects first when accessing them in the dataset/dataloader.