Compatibility of pytorch-geometric dataset created with PyG version <2.0

[animeshbchowdhury]

Hi team,

I have created a huge graph dataset (~870k graphs) consisting of circuits using PyG version 1.9 last year. It consists of .pt files which were created out of graphml files having lots of features. Unfortunately, with the version upgrade, it throws the error that pt files inside processed directory cannot be used and suggest to re-create them. This will again take huge compute time (over several weeks).

Is there any way by which pt dataset created using earlier version can be reused with new PyG version? This will be of immense help to us and re-creating again those pt files are gonna take huge time.

[EdisonLeeeee]

Hi @animeshbchowdhury

There is not a straightforward way to reuse your dataset with new PyG version without re-create them. Nevertheless, I would suggest other ways to achieve this. The key is to mitigate an older version of torch_geometric.data.Data class to a newer one:

• Step1: Keep your old PyG version (e.g., < 2.0) that dataset was created by
• Step2: Store your dataset information loaded from processed files:

dataset = torch.load(your_processed_file.pt)
# In most cases, dataset is a tuple containing (data, slices, ...)
data = dataset[0] # here data should be `torch_geometric.data.Data` class
torch.save(data.__dict__, 'data.pt')

• Step3: Upgrade your PyG version and recreate your data

from torch_geometric.data import Data
data = Data(**torch.load('data.pt'))

• Step4: Replace data in your dataset with this newly created data. NOTE: Please backup your your_processed_file.pt in advance!

previous_dataset = torch.load(your_processed_file.pt)
current_dataset  = (data, previous_data[1:])

# WARNING: Uncomment this to overwrite your older dataset
# torch.save(current_dataset , your_processed_file.pt)

• Step5: Let me know if there is something missing.