Different sizes of nodes and edges for GAT

[WangZijin98]

Hello,

I am using GATv2Conv for a RL problem. In my problem, the graph is changing dynamically, and the number of nodes and edges vary from step to step. I managed to solve the different dimension problem by padding the input nodes. However, when I want to add edge attributes I have a issue: I need to first define the edge dimension but the in my case the number of edges is constantly changing, so I cannot determine the edge dimension when creating the edge attribute layers. How can I solve this problem?

I am attaching the GAT layer structure I am using, it is from the source code of GATv2Conv.

  def __init__(
      self,
      in_channels: Union[int, Tuple[int, int]],
      out_channels: int,
      heads: int = 1,
      concat: bool = True,
      negative_slope: float = 0.2,
      dropout: float = 0.0,
      add_self_loops: bool = True,
      edge_dim: Optional[int] = None,
      fill_value: Union[float, Tensor, str] = 'mean',
      bias: bool = True,
      share_weights: bool = False,
      **kwargs,
  ):
      super().__init__(node_dim=0, **kwargs)

      self.in_channels = in_channels
      self.out_channels = out_channels
      self.heads = heads
      self.concat = concat
      self.negative_slope = negative_slope
      self.dropout = dropout
      self.add_self_loops = add_self_loops
      self.edge_dim = edge_dim
      self.fill_value = fill_value
      self.share_weights = share_weights

      if isinstance(in_channels, int):
          self.lin_l = Linear(in_channels, heads * out_channels, bias=bias,
                              weight_initializer='glorot')
          if share_weights:
              self.lin_r = self.lin_l
          else:
              self.lin_r = Linear(in_channels, heads * out_channels,
                                  bias=bias, weight_initializer='glorot')
      else:
          self.lin_l = Linear(in_channels[0], heads * out_channels,
                              bias=bias, weight_initializer='glorot')
          if share_weights:
              self.lin_r = self.lin_l
          else:
              self.lin_r = Linear(in_channels[1], heads * out_channels,
                                  bias=bias, weight_initializer='glorot')

      self.att = Parameter(torch.empty(1, heads, out_channels))

      if edge_dim is not None:
          self.lin_edge = Linear(edge_dim, heads * out_channels, bias=False,
                                 weight_initializer='glorot')
      else:
          self.lin_edge = None

      if bias and concat:
          self.bias = Parameter(torch.empty(heads * out_channels))
      elif bias and not concat:
          self.bias = Parameter(torch.empty(out_channels))
      else:
          self.register_parameter('bias', None)

      self.reset_parameters()

[rusty1s]

Can you clarify for me? PyG has no problem to operate on varying number of nodes or edges. It looks like your issue stems from changing feature dimensionalities. Is this correct? In that case, padding is a viable option.

[WangZijin98]

How to operate on varying number of nodes and edges? Can I use mini-batch computation by wrapping each data object into a batch via torch_geometric.data.DataLoader?
My second concern is regarding the edge attribute input. As the following code shows:
if edge_dim is not None:
self.lin_edge = Linear(edge_dim, heads * out_channels, bias=False,
weight_initializer='glorot')
else:
self.lin_edge = None
The edge dimension needs to be first claimed to perform transformation on input edge attributes. But with a varying edge number, how can I determine the edge dimension?
Thank you.

[rusty1s]

Can I use mini-batch computation by wrapping each data object into a batch via torch_geometric.data.DataLoader?
Yes.

Note that edge_dim refers to the number of features per edge, not the total edge count.