For unseen data inference with graph sage conv in inductive setting.

[Jeriousman]

I wonder how inductive setting for unseen data inference with graph sage works. The paper said it does work in an inductive setting but does just putting data into sage conv make it inductive already? Like we do everything as we normally do like in transductive way with one big graph (with neighbor sampling and mini-batching) into sage conv makes it already inductive so that I can put any unseen nodes to the model afterwards?

Or should I make input data like 'indutive data'? I mean, should I make input data inductive with subgraph for each computation graph so it is not one big graph anymore but computation graphs for all destination nodes? Then can the sage conv finally work like inductive one as it says in the paper?

I hope I am clear on what I am asking. Thank you.

[rusty1s]

The inductive setting is independent from the utilized GNN operator. You can apply SAGEConv in both transductive and inductive learning scenarios. For an inductive learning scenario, you will have one graph to train on, and one (or more) graphs to test on, e.g., train_data and test_data. You can then train your model on train_data and appy it on test_data, e.g.:

train_loader = NeighborLoader(train_data, ...)
train(model, train_loader)

test_loader = NeighborLoader(test_data, ...)
test(model, test_loader)

[Jeriousman]

Thank you for your answer. If I do NeighborLoader with a big graph for train and test loaders, how can I then ensure train_data, test_data are independent (no edges between them)? I guess it is my job to do so before I do the neighborloader?

[rusty1s]

If you want to do it, it's indeed your job. Importantly, you do not have to do this. You are well allowed to make use of node features of nodes not in your training set during feature propagation, as long as you only make use of ground-truth labels inside your training set.

If you still want to apply inductive learning, you will need to split your data beforehand:

train_data = data.subgraph(data.train_mask)
train_loader = NeighborLoader(train_data, num_neighbors=..., ....)

Let me know if this clarifies things :)

[Jeriousman]

@rusty1s You mentioned this
You are well allowed to make use of node features of nodes not in your training set during feature propagation, as long as you only make use of ground-truth labels inside your training set.
But in CS224W (https://www.youtube.com/watch?v=ewEW_EMzRuo&list=PLoROMvodv4rPLKxIpqhjhPgdQy7imNkDn&index=25), Jure Leskovec mentioned that in inductive data split, you should cut off all edges between train, valid and test sets. And also nodes must be only within their own datasets respectively. So now I am confused with your reply. Could you elaborate your statement?

[rusty1s]

Yes, in that case, you would need to apply data.subgraph beforehand. My answer was related to the transductive setting, which seems much more common for node classification. If you have the data, there's no real reason to not use it :)

[pigu163]

@rusty1s Hi, assume that there are two graph, one for training, one for testing, train a graphSAGE on NeignborLoader(data, num_neighbors = [15, 10], batch_size = 128).
During inference, do I need also this kind of NeignborLoader ? Do i need ensure that during inference, each target node have same num_neighbors as training NeignborLoader ?

[pigu163]

So far, I am using NeignborLoader for training a GraphSAGE, but use whole graph on inference(don't subsample), the result have big gap while changing the random seed, look like the GraphSAGE can not generalize well

[rusty1s]

In general, you can use different sampling parameters/full neighborhood as long as your aggregation is independent of the number of neighbors. That holds for, e.g., mean, max or attention, while sum would yield embeddings which are out of distribution.