global add pool when num features per node is 1

[mdanb]

Does it make sense to do a global add / mean / max pool when I only have 1 resulting feature per node (i.e node embedding dim = 1) for graph classification? Or should I just concatenate all of the features into one vector?

[rusty1s]

This should work just fine. You need some way to aggregate the node features in a permutation-invariant fashion, so I do not recommend using concatenation.

[rusty1s]

You can ignore it in that case.

[mdanb]

PS: I tried comparing pooling to concatenation. When I pool (using smthn like add or sum), the model is incapable of learning (just predicts majority class). When I don't, it is capable of learning. Might there be a theoretical justification for that? Or perhaps just a heuristic explanation? I'm thinking (and would love to here your thoughts on this) that this might indicate that a graph based model is not necessarily suited for the task? I say this because if I concatenate all node features, then the model starts to look to me a lot like just a simple feedforward NN, with a bit of message passing in the beginning (I did my experiments using SGC). If I do concatenation with SGC with message passing vs no message passing (i.e K=0, and K=2 say), then the performance is similar and the model learns in both cases. However, if I pool instead of concatenating, then it learns nothing.

[rusty1s]

Ensuring permutation-invariance leads to overall functions that are inherently constrained. In general, this is a useful property for working on graph-structured data, but also comes with some downsides, e.g., all neighbors are transformed the same. In your case, such a constraint might lead to poor performance since you do not have access to rich node feature information. When working on static graphs, permutation-invariance is usually not mandatory (since your graph ordering is fixed anyway), so your model can learn a richer class of functions. However, the number of model parameters now depend linearly on the number of nodes in your graph (which might not scale to larger graphs).

You are right with your comparison to simple feedforward NNs. Overall, I think in your case it is reasonable to expect that concatenation works better than pooling. There are also some pooling layers that try to imitate such a behavior in a permutation-invariant fashion, e.g., torch_geometric.nn.global_sort_pool.

[mdanb]

I see. But then is there any advantage to using a GNN compared to simple feedforward NNs in this case?
Thanks a lot for the feedback!

[rusty1s]

That depends on the task I guess. Here, the GNN is used to enhance features representations by aggregating neighbor information, which might be useful to de-noise some node features. I think it might be more useful if you increase its feature dimensionality in order to learn some graph-specific patterns, e.g.:

lin = Linear(1, 64)
conv = GCNConv(64, 64)

x = lin(x).relu_()
x = conv(x, edge_index).relu_()