How to add a virtual node?

[finquick]

When I did the graph classification task, I wanted to add a virtual supernode to represent the whole graph features. But when miniBatch is used, the shape of the data changes. For example, if the dim is 32*9*128 , 32 is batchsize, 9 is nodenum, and 128 is the channel dim. Assuming the number of channels remains the same, the final feature x will be 288*128. I have no idea how to add virtual nodes, should I add a node feature with all zero for each graph when I need to build the dataset? or cat columns of zero?Just like this：

import torch
from torch_geometric.data import Data
from torch_geometric.nn import GATConv

edge_index = torch.tensor([[0, 1, 1, 2, 1],
                           [1, 0, 2, 1, 3]], dtype=torch.long)
x = torch.randn(3, 16)
x = torch.cat([x, torch.zeros(16).reshape(1,-1)],dim=0) #add virtual node

data = Data(x=x, edge_index=edge_index)

model = GATConv(16, 16)

out = model(data.x, data.edge_index)

If in a batch，get the supernode feature like this:

x = torch.randn(32*10, 16)
result = x[9::10]

right?
Another problem was that I wanted to mix the GCN and CNN layers. Using MiniBatch would lower the dimensions compared to PyTorch,For example, it is 32*9*128 in pytorch, and the feature dimension of x after calculation in PYG will become (32*9=288)*128. If I want to send the result of GCN to CNN during use, I only need to reshape the feature into 32*9*128 doesn't have any effect, does it?
Thanks!

[rusty1s]

There are two ways to add virtual nodes:

1. As you described by adding a node to x (a zero feature vector). Keep in mind that you also need to update edge_index such that the virtual node is connected to all other nodes.
2. You can also do virtual node message passing outside of a GNN layer, e.g.

x_global = global_mean_pool(x, batch)
x = x + self.lin(x_global)[batch]

Yes, reshaping is all you need to do.

[finquick]

Yeah, great!
In my opinion, the second way is not as good as the first way to allow virtual nodes to message according to human prior knowledge, similar to a readout layer just for message aggregation! My intuition is that the first method may be more effective, and I will test it with experiments.
Thank you!

[rusty1s]

I think they are equally effective. I will also work on adding a VirtualNode transform in PyG to make the construction of virtual nodes more intuitive. I will keep you posted.

[rusty1s]

Added a first working version here: #4163
Let me know if that is useful to you!