How to use a custom dataset for Heterogeneous Graph Learning ?

[Justsecret123]

Following the tutorial on Heteregeneous Graph Learning, I created my own dataset using Networks graphs as follows :

dataset = []
for key, value in test.items():
    print(key, end="\r")
    # New graph 
    graph = from_networkx(value)
    #graph = HeteroData(graph)
    dataset.append(graph)

p_dataset = MyDataset(root="../Scripts/pytorch_dataset",data_list=dataset)

After successfully loading the created dataset using data=torch.load("../Scripts/pytorch_dataset/processed/dataset_pytorch_obj.pt"), I cannot seem to understand how can I get the metadata from the loaded dataset, at this step :

model = GNN(hidden_channels=64, out_channels=4)
model = to_hetero(model, data.metadata, aggr='sum')

It just returns an error : AttributeError: 'tuple' object has no attribute 'metadata'.

How can I get the metadata from the generated .pt file ?

Also, how can I define the x_dict since it also returns an error ? Should I modify the original Nx graphs to contain a feature named "x", and how ?

Edit : Adding the "x" feature to the nodes on my original nx graphs produced no effect on the outcome, the x_dict remains empty.

[rusty1s]

First of all, it looks like the data from disk is stored as a tuple (not sure why), so you may want to unpack it if you are only interested in the Data object, e.g., torch.load(path)[0]. Second, note that from_networkx does not support heterogeneous graphs yet (there is an open issue about this), and so the usage of heterogeneous GNNs is not applicable here. You can get around this by manually creating a HeteroData object if you definitely need heterogeneous GNNs.

[Justsecret123]

This is the dataset loaded from disk :

Where should I get the edge index from ? Moreover, there's no x feature on data[0] nor data[1], which looks strange since all my graphs possess that feature.

I think we would benefit from a complete example with a custom dataset (In Memory?) loaded from disk/from a list object.

[rusty1s]

Can you share some more details on how you processed the data? If the data does have node features before saving it, it should also have node features after loading it.

[Justsecret123]

My original Networkx.Digraph nodes contain three features : x, type, and value. Before saving the list of graphs as a (InMemory)dataset, I can retrieve the x feature of a specific graph by accessing the list at a specific index.

Unfortunately, when loading the dataset from disk, that particular x feature just seems to be left out of the equation. The definition of the MyDataset class is as follows :

class MyDataset(InMemoryDataset):
    def __init__(self, root, data_list, transform=None):
        self.data_list = data_list
        super().__init__(root, transform)
        self.data, self.slices = torch.load(self.processed_paths[0])

    @property
    def processed_file_names(self):
        return "dataset_pytorch_obj.pt"

    def process(self):
        torch.save(self.collate(self.data_list), self.processed_paths[0])

[rusty1s]

Can you share a minimal example to reproduce? This would help me to track down the issue as otherwise I am out of ideas.

[Justsecret123]

As I cannot share data related to the project at the moment, it would be better if I provide some cues about how I generate the graphs instead.

Starting from a dictionary object, I create the nodes and the edges as follows (ex: Emotion nodes) :

# Generating the Emotion node
scene_graph.add_node(emotion, x=emotion, type="Emotion", value="")
# Generating the edge from Character to Emotion
scene_graph.add_edge(character, emotion, type="expresses", role="")

The here obtained graphs and their structure have already been verified and used with other libraries, the problem doesn't seem to come from here.

Then, the list of graphs is fed by looping over the generated graphs, then applying the from_networkx function to each graph and finally appending it to the list.

After this process, I am able to retrieve the x feature on each graph (ex: data[1].x). Unfortunately, after casting that list into a PyTorch dataset, that feature just disappears.

I could have used the list object as it if there was a tutorial or guide using a list of graphs as the starting point of a node/graph classification problem.