Graph regression

[mtzortzi]

Hi,

Thank you for all the work and help that you offer to all the community.

I'm having a dataset of graphs, which looks like this: Data(edge_attr=[444, 1], edge_index=[2, 444], x=[251, 4], y=[2]).
The node attributes are 4 as I wrote, but some of them are the same for all the nodes. This is because I had some features characterizing the whole graph (graph data) and I thought to add it to each node attribute so as to use pytorch geometric.

My goal is to predict those 2 values in y, which characterize the whole graph by taking some X graphs and predicting the y for the next X+3 graphs (3 is an example). I have divided my pytorch_geometric graphs into train, validation, and test sets and I have written some code using graph transformers.

But I'm a bit confused about how to approach the problem as long as the dataset is concerned. I suppose it's a regression problem and that the loss function I should use is MSE or MAE loss.
One of my ideas is to create masked values for y (for the next graphs of which the prediction I want), but in this case, what should I do with the edge_attr, edge_index, and x values of these graphs and their training part?

Could you please enlighten me with which approach I should tackle this problem?

Thank you for your time,
Marianna

[rusty1s]

Interesting problem. You could try to implement a dataset wrapper class, that returns 3 graphs and the labels of the fourth:

class GroupDataset(torch.utils.data.Dataset):
    def __init__(self, dataset, window=3):
        super().__init__()
        self.dataset = dataset
        self.window = window
        
    def __len__(self):
        return len(self.dataset) - self.window
        
    def __getitem__(self, idx):
        data_list = [self.dataset[idx + offset] for offset in range(self.window)
        y = self.dataset[idx + self.window].y
        return data_list, y

[mtzortzi]

Well, my initial tensors are e.g.: x of shape torch.Size([17344, 4]) and edge_attr torch.Size([28320,1]).

For the embeddings I use

self.embedding_h = torch.nn.Embedding(n_input_dim+1, hidden_dim)
self.embedding_e = torch.nn.Embedding(e_input_dim+1, hidden_dim)

where n_input_dim is the size of the dictionary of embeddings (number of nodes or edges)
and hidden_dim is the size of each embedding vector.
So when I call the embedding from the forward function

h = self.embedding_h(x)

to calculate the embeddings the resulting tensors for each of my input tensors are:
for node embeddings torch.Size([17344, 4, 64]) and
for edge embeddings torch.Size([28320, 1, 64]).

These are the tensors that are passed into the TransformerConv.

[rusty1s]

I see. You should either reshape them to torch.Size([17344, 256]) and torch.Size([28320, 64]), or sum them up in the second dimension.

[mtzortzi]

Thank you for your instructions. I had already tried the reshape with .view() and it doesn't work inside the Transformer Conv due to dimensional conflicts, but the summation in the second dimension works fine and I didn't think about it before. As I'm interested in inserting positional encoding and other types of encoding inside the TransformerConv layer, I would like to ask you whether this could be possible.

[rusty1s]

Well, this depends on how you define positional encodings in a graph :) There exists some work of that in the field, e.g., here, but this isn't yet integrated into PyG.

[mtzortzi]

Yes, you're right. I'll have a look and I'll come back to you. Thank you again for your help.