PNA without edge_attributes

[AdarshMJ]

Hi! So Im trying to use - https://github.com/pyg-team/pytorch_geometric/blob/master/examples/pna.py on a custom dataset, but my dataset doesn't have edge_attr. Is there a way to generate dummy edge_attr to use it with the code ? Any suggestions would be welcome. Thank you in advance!

[rusty1s]

No need to create dummy edge features. Simply drop the edge_dim argument and do not pass edge_attr to the forward call of the GNN layer.

[AdarshMJ]

Hi! Thank you for the response. It works. But I was having a runtime error. Im not sure how to resolve this. Could you please point me in the direction as to where could be the problem. Thanks

[rusty1s]

How does your data look like? What‘s the output of print(data) and print(data.edge_index.max())?

[AdarshMJ]

Hi!
It looks like this -
data DataBatch(x=[655, 11], edge_index=[2, 1220], edge_attr=[1220, 4], y=[64, 19], pos=[655, 3], idx=[64], name=[64], z=[655], batch=[655], ptr=[65]) EDGEINDEX tensor(654)

[rusty1s]

Mh, this looks correct. Does the problem also exist for other GNN layers? Can you show me the full traceback?

[AdarshMJ]

`data DataBatch(x=[646, 11], edge_index=[2, 1202], edge_attr=[1202, 4], y=[64, 19], pos=[646, 3], idx=[64], name=[64], z=[646], batch=[646], ptr=[65])
EDGEINDEX tensor(645)

RuntimeError Traceback (most recent call last)
in ()
1 for epoch in range(1, 301):
----> 2 loss = train(epoch)
3 # val_mae = test(test_loader)
4 test_mae = test(test_loader)
5 scheduler.step(test_mae)

4 frames
in train(epoch)
8 print("EDGEINDEX",data.edge_index.max())
9 optimizer.zero_grad()
---> 10 out = model(data.x, data.edge_index, data.batch)
11 loss = (out.squeeze() - data.y).abs().mean()
12 loss.backward()

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1100 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1101 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102 return forward_call(*input, **kwargs)
1103 # Do not call functions when jit is used
1104 full_backward_hooks, non_full_backward_hooks = [], []

in forward(self, x, edge_index, batch)
49 x = self.node_emb(x.squeeze().long())
50 for conv, batch_norm in zip(self.convs, self.batch_norms):
---> 51 x = F.relu(batch_norm(conv(x, edge_index)))
52
53 x = global_add_pool(x, batch)

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1100 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1101 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102 return forward_call(*input, **kwargs)
1103 # Do not call functions when jit is used
1104 full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.7/dist-packages/torch_geometric/nn/conv/pna_conv.py in forward(self, x, edge_index, edge_attr)
153 x = x.view(-1, self.towers, self.F_in)
154 else:
--> 155 x = x.view(-1, 1, self.F_in).repeat(1, self.towers, 1)
156
157 # propagate_type: (x: Tensor, edge_attr: OptTensor)

RuntimeError: shape '[-1, 1, 75]' is invalid for input of size 177650`

[AdarshMJ]

Yes Im trying to train it against target 0. Im normalising as follows -

mean = dataset.data.y[:, target].mean().item()
std = dataset.data.y[:, target].std().item()
dataset.data.y[:, target] = (dataset.data.y[:, target] - mean) / std
error += (model(data) * std - data.y * std).abs().sum().item()  # MAE```

[AdarshMJ]

But the MAE is still high

[rusty1s]

Thanks for confirming. It looks like something different is causing this. How does your final model architecture look like? Did you try to tune hyperparams such as the learning rate?

[AdarshMJ]

Hi Yes, I tried a couple of LRs, my final architecture looks like this -

class Net(torch.nn.Module):
    def __init__(self):
        super().__init__()

        #self.node_emb = Embedding(21, 11)
        #self.edge_emb = Embedding(4, 50)

        aggregators = ['mean', 'min', 'max', 'std']
        scalers = ['identity', 'amplification', 'attenuation']

        self.convs = ModuleList()
        self.batch_norms = ModuleList()
        for _ in range(4):
            conv = PNAConv(in_channels=dataset.num_features, out_channels=11,
                           aggregators=aggregators, scalers=scalers, deg=deg,
                           towers=11, pre_layers=1, post_layers=1,
                           divide_input=False)
            self.convs.append(conv)
            self.batch_norms.append(BatchNorm(11))

        self.mlp = Sequential(Linear(11, 64), ReLU(),Linear(64, 19))

    def forward(self, x, edge_index, batch):
        #x = self.node_emb(x.squeeze())
        #edge_attr = self.edge_emb(edge_attr)

        for conv, batch_norm in zip(self.convs, self.batch_norms):
            x = F.relu(batch_norm(conv(x, edge_index)))

        #x = global_add_pool(x, batch)
        x = global_add_pool(x, batch)
        return self.mlp(x)```

[AdarshMJ]

And changing the MAE loss to use mean instead of sum gives -
total_error += (out.squeeze()*std - data.y*std).abs().mean().item()

[rusty1s]

Note that your loss should be averaged across all examples (otherwise the loss value gets increased if the size of your dataset gets bigger). Currently, it looks like you are only averaging within mini-batches.

Your model architecture looks good to me, but I would probably start benchmarking with towers=1 and a reasonable hidden_channels value, e.g., 64 or 128.

[AdarshMJ]

Im sorry I dont fully understand,

in the train function, there is this line which means its getting averaged over all the examples right?
total_loss += loss.item() * data.num_graphs
But in the test function, the loss calculated has an option to either use the
total_error += (model(data) * std - data.y * std).abs().sum().item() # MAE
or
total_error += (model(data) * std - data.y * std).abs().mean().item() # MAE

If I use mean, the values are in the order of 1.78 and if I use sum, the values are really large like 300 etc.

[rusty1s]

Using sum is correct here. At the end, you need to divide by the number of graphs.

[AdarshMJ]

Thank you :)