GAT Shape Mismatch with Sparse Matrices

[hadarshavit]

I'm using GAT with sparse matrix for the edges. The each edge has 1 attribute.
I call to the layer in the following way:

x, adj_t = data.x, data.adj_t
x = layer(x, adj_t)

Where data is a data batch.
However I get the following error:

  File "......python3.10/site-packages/torch_geometric/nn/conv/gat_conv.py", line 230, in forward
    edge_index = set_diag(edge_index)
  File "......python3.10/site-packages/torch_sparse/diag.py", line 63, in set_diag
    new_value[inv_mask] = torch.ones((num_diag, ), dtype=value.dtype,
RuntimeError: shape mismatch: value tensor of shape [4436] cannot be broadcast to indexing result of shape [4436, 1]

The shape of x is torch.Size([4436, 160]) and adj_t is:

SparseTensor(row=tensor([   0,    0,    0,  ..., 4435, 4435, 4435], device='cuda:0'),
             col=tensor([   8,   11,   14,  ..., 4433, 4433, 4434], device='cuda:0'),
             val=tensor([[ 1.],
                           [-1.],
                           [ 1.],
                           ...,
                           [ 1.],
                           [-1.],
                           [-1.]], device='cuda:0'),
             size=(4436, 4436, 1), nnz=55032, density=0.28%)

When creating the Data object I'm sending a tensor of size [num_edges, 1] as the edge_attr, and then use the ToSparseTensor transform.

[rusty1s]

I am sorry to say, but GATConv with a SparseTensor and edge features is currently not yet supported, see https://github.com/pyg-team/pytorch_geometric/blob/master/torch_geometric/nn/conv/gat_conv.py#L239. You have to use edge_index format.

[hadarshavit]

Thanks, however when I'm using GCNConv I also have an error:

  File "python3.10/site-packages/torch_geometric/nn/conv/gcn_conv.py", line 187, in forward
    edge_index = gcn_norm(  # yapf: disable
  File "python3.10/site-packages/torch_geometric/nn/conv/gcn_conv.py", line 48, in gcn_norm
    adj_t = mul(adj_t, deg_inv_sqrt.view(-1, 1))
  File "python3.10/site-packages/torch_sparse/mul.py", line 21, in mul
    value = other.to(value.dtype).mul_(value)
RuntimeError: output with shape [60918] doesn't match the broadcast shape [60918, 60918]

It works if the edge_attr has a shape of [num_edges] instead of [num_edges, 1], but I'm not sure if this is the correct way to use it.

[rusty1s]

Yes, we expect edge_weight to be a vector, so it should have shape [num_edges].