The relationship of generalizability of GNN and the dataset size

[YiKangOY]

Hi all,

Now I have a graph-level regression task. I have a dataset of 39 graphs, now I use 34 to train and 5 to test.
The model I use is GIN and I use two layers. I use a sum pooling to get the embedding of the whole graph.

The result on the test graphs is really bad. I am wondering if this dataset is too small for a graph-level task so the model generalizes badly.

[rusty1s]

This is hard to tell, your dataset looks indeed quite small. Does the test performance actually increase over the number of epochs? What happens if you add some common regularization techniques?

[YiKangOY]

I have used Batchnorm after each layer of GNN and I also used it in MLP.
Should I use Batchnorm and Layernorm simultaneously or how should I choose between them?

[rusty1s]

Mh, okay. If you are already using normalization, then this is good. No need to apply batch norm and layer norm simultaneously.

[rusty1s]

One thing you can do is to confirm that your GNN actually does something reasonable. For example, an MLP should have worse evaluation performance. It might also be that your GIN model is too expressive, and you may have better luck with alternatives such as SAGEConv.

[YiKangOY]

Thanks, I will try other networks, but what do you mean by "For example, an MLP should have worse evaluation performance."? I just use MLP to regress for the target after GNN,

[rusty1s]

Meaning an MLP that replaces the GNN.