GlobalAttentionGAT Net With Multiple Headers - RuntimeError: mat1 dim 1 must match mat2 dim 0

[frankwxu]

Error message where use Multiple Headers. I modified the code based on the tutorial.

`

Class GlobalAttentionGATNeWithHeaders(torch.nn.Module):

  def __init__(self, num_node_features, hidden_channels, num_heads):
      super(GlobalAttentionGATNeWithHeaders, self).__init__()
      self.conv1 = GATConv(num_node_features, hidden_channels, num_heads)
      self.conv2 = GATConv(num_node_features * num_heads, hidden_channels, 1)
      self.att = GlobalAttention(Linear(hidden_channels, 1))
      self.lin1 = Linear(hidden_channels, hidden_channels)
      self.lin2 = Linear(hidden_channels, num_node_features)

  def reset_parameters(self):
      self.conv1.reset_parameters()
      self.conv2.reset_parameters()
      self.att.reset_parameters()
      self.lin1.reset_parameters()
      self.lin2.reset_parameters()

  def forward(self, data):
      x, edge_index, batch = data.x, data.edge_index, data.batch
      x = F.relu(self.conv1(x, edge_index))
     // probelm is here
      x = F.relu(self.conv2(x, edge_index))
      x = self.att(x, batch)
      x = F.relu(self.lin1(x))
      x = F.dropout(x, p=0.5, training=self.training)
      x = self.lin2(x)
      return x`

Here is the error message

File "C:\Users\xxx\PycharmProjects\test_torch_env\pkgcode2vec\mynn.py", line 209, in forward x = self.conv2(x, edge_index) File "C:\Users\xxx\anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "C:\Users\xxx\anaconda3\envs\pytorch\lib\site-packages\torch_geometric\nn\conv\gat_conv.py", line 125, in forward x_l = x_r = self.lin_l(x).view(-1, H, C) File "C:\Users\xxx\anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "C:\Users\xxx\anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\linear.py", line 93, in forward return F.linear(input, self.weight, self.bias) File "C:\Users\xxx\anaconda3\envs\pytorch\lib\site-packages\torch\nn\functional.py", line 1692, in linear output = input.matmul(weight.t()) RuntimeError: mat1 dim 1 must match mat2 dim 0

[wsad1]

I guess, the input dimension to self.conv2 should be hidden_channels*num_heads.
self.conv2 = GATConv(hidden_channels * num_heads, hidden_channels, 1).

[frankwxu]

Thanks. I assume that the actual output of the conv1 is a dimension of 'hidden_channels*num_heads' already due to the multiple heads. How to fix it, please?

[wsad1]

@frankwxu , so you made the fix i suggested but you are still getting some error. Is my understanding correct. Also could you please elaborate your current problem more, that should make it easier to suggest fixes.

[frankwxu]

Sorry for the confusion:

1. my program works correctly without using the "num_heads" parameter', e.g., default is 1
2. After adding "num_heads" parameter, I (ONLY) changed the line from

self.conv2 = GATConv(num_node_features, hidden_channels)
to
self.conv2 = GATConv(num_node_features * num_heads, hidden_channels, 1)

and I have the same error.

3. I add a breakpoint, I observed that the 'self.conv1' produces the correct dimensions, which is 'num_node_features * num_heads'

[wsad1]

Let me address points 2 and 3.
2. The change to self.conv2 is in the first argument input_channels, so it should be self.conv2 = GATConv(hidden_channels * num_heads, hidden_channels)
3. Correct me if I am wrong. But the output of self.conv1, based on the code you shared, should have shape hidden_channels*num_heads or just hidden_channels if num_heads=1 and not num_node_features*num_heads, right?