Creating my Own Dataset

[anthonysirico]

Hello, I just have a question on how to create my own dataset. What i have is a MATLAB .mat file which contains the following struct:

A = each cell contains an adjacency matrix
NL = each cell contains an array with each nodes label which is a string, i.e. [C, C, R, G, O]. Yes, there are nodes with the same label in a single graph.
Label = each cell is the graph label which is a float representing a "performance" value

That is it, i do not have any more features then that; 43,000 graphs, node labels, and a corresponding graph label.

My original path was to run a for loop and convert each graph to networkx:

nx_graph = []
for i in range(len(A)):
     nx_graph.append(nx.Graph(A[i]))

And then ord() the node labels and use them as features.

for i in range(len(nx_graph)):
     for j in range(len(nx_graph[i].nodes)):
          nx_graph[i].nodes[j]['feature'] = [ord(NL[i][j])]

And then of course have a separate series or list for the graph labels.

Then I would perform a for loop and convert each using torch_geometric.utils.from_networkx

tg_graph = []
for i in range(len(nx_graph)):
     tg_graph.append(from_networkx(nx_graph[i]))

From the data that i have, is this the best way to do that? I want the best format for my data to have PyG be successful in this endeavor, because Stellargraph failed in this attempt. I am trying to perform a regression where x = a graph, and y = the label. If i have to do 2 classes thats fine, but i would like to attempt this first. Please help! Thank you in advance.

[rusty1s]

One issue with the current approach is that you are using integers as features, while NNs require features of floating point type. As such, I recommend to convert your feature index to a one-hot representation via torch.nn.funxtional.one_hot. You can do this after conversion to PyG. Let me know if this helps!

[anthonysirico]

I dont know if one_hot would work. The number of nodes in the graphs range from 6 to 20.

[rusty1s]

Yes, indeed. You may want to specify the num_classes argument to ensure that the list lf every node in every graph is equally long.

[anthonysirico]

So I think i figured out the first challenge

def graph_data(A, X, labels):

    tg_graphs = []

    for i in range(len(A)):
        coo = coo_matrix(A[i])
        indices = np.vstack((coo.row, coo.col))
        x = [ord(i) for i in X[i]]
        index = torch.LongTensor(indices)
        feature = torch.FloatTensor(x)

        graph = Data(x=feature, edge_index=index, y=labels[i])

        tg_graphs.append(graph)

    return tg_graphs

Where tg_graphs is:

[Data(x=[18], edge_index=[2, 42], y=0.0912047881539918),
 Data(x=[18], edge_index=[2, 42], y=0.0911561592893587),
 Data(x=[17], edge_index=[2, 40], y=0.0939426626061364),
 Data(x=[17], edge_index=[2, 40], y=0.0911476283054859),
 Data(x=[18], edge_index=[2, 42], y=0.0911682263035731),
 Data(x=[19], edge_index=[2, 40], y=0.0885647857159851),
 Data(x=[19], edge_index=[2, 40], y=0.0901584221915995),
 Data(x=[19], edge_index=[2, 40], y=0.0901584221916017),
 Data(x=[19], edge_index=[2, 40], y=0.0716963824344366),
 Data(x=[20], edge_index=[2, 42], y=0.0786077013779856),
 Data(x=[19], edge_index=[2, 40], y=0.0946293611737009),
 Data(x=[20], edge_index=[2, 42], y=0.0691385630931132),
 Data(x=[18], edge_index=[2, 38], y=0.0939426621168369),
 Data(x=[19], edge_index=[2, 40], y=0.0805215970455449),

Is there a way to convert this to a dataset that i can save and not have to constantly do this?

[rusty1s]

Yes, you can store your objects via torch.save, or use the InMemoryDataset wrapper of PyG.

[anthonysirico]

I have been trying to use InMemoryDataset and I'm sorry i am having a little trouble following the instructions. Can you give me an example of how to do it? Danke!

[rusty1s]

Did you look at the tutorial in the documentation? There also exists a lot example usages in torch_geometric/datasets (e.g., datasets/faust.py). In general, you can do the processing in process(), save the data at the end of the function, and load it in once again at the end of __init__.