Input for GCNConv layer (torch.sparse coo vs edge index and edge attr tensors)

[charleneolive]

Hello,

I am slightly confused by the behaviour of GCNConv.
When I used the input as a torch.sparse_coo tensor vs having the edge_index and edge_attr placed as separate inputs, I get a different outcome from the gcnconv operation. Debugging it made me conclude that it was primarily due to the coalesce function, which has been applied to the torch.sparse_coo tensor but not for the scenario where I used edge_index and edge_attr as inputs. In particular, this happened during the add_self_loops_fn for the torch_sparse_coo_tensor, under to_torch_coo_tensor.

I was trying to debug this as I was training a GCN model where the performance decreases when I used inputs as edge_index and edge_attr, instead of using as a sparse tensor.

For a reproducible scenario, please refer to the collab link here (under Differences in output):
https://colab.research.google.com/drive/144kSk-OfSwZHChfnSHbpmB-iV_4ssCZY?usp=sharing

This is based on the example here: https://pytorch-geometric.readthedocs.io/en/latest/get_started/introduction.html

Would really appreciate if someone can enlighten me on the following

1. What is the purpose of coalesce and why is it not used for the other type of input?
2. How would it affect the performance when training the GCN?
3. What should be the appropriate inputs to the GCNConv? Can I use torch.sparse_coo_tensor? How can I get the same outcomes regardless of the inputs I used?

[rusty1s]

Note that for non-symmetric sparse tensors, you need to input the transposed adjacency matrix:

x_out = conv1(x, ori_edge_index, edge_weight)
tensor_adjacency = to_torch_coo_tensor(data.edge_index, edge_weight)
x_out2 = conv1(x, tensor_adjacency.t())
print(torch.allclose(x_out, x_out2, atol=1e-4))

works for me.

[charleneolive]

Thank you so much for your quick reply!
I tried it out and met with an error

Upon adding coalesce, the error is resolved:

Could I understand a bit more about why coalesce is required for the adjacency matrix?

[EdisonLeeeee]

Hi @charleneolive

Coalescing is used to eliminate duplicate coordinates in the indices of a sparse adjacency matrix, which is necessary when accessing the indices() and values().

FYI: https://pytorch.org/docs/stable/sparse.html

[charleneolive]

Hello @EdisonLeeeee,
Thank you so much. It makes sense now.

Could I ask if there are any considerations otherwise? Because when I tried it on my dataset, I still face a problem where they are not similar:

Unfortunately, I cannot share the dataset due to data privacy considerations. Perhaps if there's any advice what should we look out for to make sure the the values are similar? Thank you so much!

[charleneolive]

Hello @rusty1s ,

I have checked and my pyG version is 2.3.1. Based on the documentation from 2.3.1, edge weights and edge indices are supported: https://pytorch-geometric.readthedocs.io/en/2.3.1/generated/torch_geometric.nn.conv.GCNConv.html?highlight=gcnconv#torch_geometric.nn.conv.GCNConv.

For the adjacency matrix, it wasn't mentioned explicitly, but it was also able to get accepted, and under the gcn_norm in GCNConv, it passes through "if is_torch_sparse_tensor(edge_index)"

Currently, the inputs of edge weights and edge indices show very poor convergence for the training dataset and a poor performance overall for the test dataset, while for the adjacency matrix, it shows much better convergence.

[rusty1s]

Just confirmed that it works fine for me on 2.3.1 as well.

Currently, the inputs of edge weights and edge indices show very poor convergence for the training dataset and a poor performance overall for the test dataset, while for the adjacency matrix, it shows much better convergence.

That's interesting, but I don't have a good explanation for this yet. If you ever manage to come up with a reproducible example, I am very curious to look into this.

[charleneolive]

I realised that the difference is caused by self loops. If I pass add_self_loops = False into GCNConv, I get the same outputs regardless of what type of input I used. I have also validated this by comparing the edge index and weights before and after adding self loops. For the adjacency matrix, the function used for adding self loops is add_self_loops_fn while for the inputs of edge_weight and edge_index, the function is add_remaining_self_loops

[charleneolive]

To add on to my previous comment, when I retrained my model again with add_self_loops=False for both using adjacency matrix and edge_index and edge_weights, I got the same result.

[charleneolive]

It's alright! I think I got it.
As for the adjacency matrix the repeated self-loops are not removed, this would cause two self-loops, as self-loops are already present in the adjacency matrix. For this dataset, maybe a stronger emphasis on the features of the node itself is required, and maybe works better in this scenario (please correct me if my understanding is wrong).