Performing inference with a trained model on heterogeneous data

[Tom433]

I'm currently working with a graph model that involves heterogeneous graphs and node classification for a specific node type. I haven't been able to find any examples or documentation on how to perform inference on this type of model after it has been trained.

During the inference process, new nodes of various types are introduced, and I'm particularly interested in understanding how to utilize the Neighborhood Loader in this context. I would greatly appreciate any guidance or resources that can help me understand how to conduct inference on a trained graph model, especially when dealing with the addition of new nodes during inference time.

[rusty1s]

In general, GNNs are inductive, so there is not a problem with applying a trained graph model on unseen graphs or nodes. In the PyG case, you can just apply your model on new or updated data objects, e.g.:

out = model(new_data.x, new_data.edge_index)

If you want to leverage the NeighborLoader, you can create a new instance for which you only compute representations for newly added nodes, e.g.:

loader = NeighborLoader(new_data, input_nodes=new_node_ids, ...)

...

Let me know if this answers your question.

[Tom433]

What, in your opinion, is the preferred approach in such a case?

[rusty1s]

What do you mean?

[Tom433]

If the graph is too large to perform the complete inference, and I don't want to initialize the NeighborLoader each time, is there an alternative way? One possibility is to use a subgraph, I suppose.

[rusty1s]

Yes, you would just run your GNN on the subgraph you are interested about, the question is how to obtain this subgraph (e.g., via NeighborLoader, k_hop_subgraph or something else).

[Rico2000]

@rusty1s So there is no efficent way with the Loaders in pyg todo inference on a Trained GNN with a Graph that is to big to fit in Memory? The only Option ist k_hop_subgraph or figure out another Strategy to get a smaller Subgraph? Thanks fo your help

[akihironitta]

@Rico2000 You can still use NeighborLoader or k_hop_subgraph if it's just that your entire graph cannot fit in GPU memory. As far as I understand, the discussion above is about an issue when your graph changes frequently (e.g. when new nodes are added to your graph).

[Rico2000]

@akihironitta yes i understand but if i want to use GNN in a production setting where i have to classify new unseen nodes and the udnerlying Graph is to big to do inference on the complete graph. There is no Option for me to do that?

[rusty1s]

Our samplers currently work on the CSR representation of the adjacency matrix to guarantee O(1) neighborhood access. The problem is that this representation is hard to extend without duplicating memory. So currently, the only workaround is to pre-process the subgraphs you want to predict on offline, or re-initialize the graph/loader.