GraphGym: Evaluate on a set of graphs for node classification

[yulonglin]

On GraphGym, I hope to train on a graph, and evaluate on a separate graph. They will have the same input/output space (i.e. same number of node features), and I want to see how my model generalises to another graph.

How should I approach the data splitting and training code? This is quite a general question, but I'm a bit stuck and am starting from a codebase that uses GraphGym: https://github.com/luis-mueller/probing-graph-transformers

UPDATE: One potential option I'm looking into is modifying the create_loader function in GraphGym directly.

[rusty1s]

Mh, yeah I assume you are right because GraphGym either supports training on a single graph for node classification, or on a set of different graphs for graph classification. IMO, the most easiest approach would be to utilize the trained model and run that on a different graph, e.g., by simply doing

train(model, datamodule)
model(new_data)

[yulonglin]

Thanks! That makes sense. What I did in the end involved creating a new type of task that creates data loaders just like graph tasks, and then modifyGNNNodeHead's _apply_index() to ignore masking. I was thinking it might make sense to add similar functionality to the main GraphGym codebase since it makes sense for certain robustness and generalization experiments, but what you're suggesting may be more straightforward and simple to do