NeighborLoader

[Chrisable]

when i see this code '' https://github.com/pyg-team/pytorch_geometric/blob/master/examples/reddit.py'',I found that NeighborLoader use data instead of train_data.Why? If so,the sampled neighbors may have nodes in the test set.

train_loader = NeighborLoader(data, input_nodes=data.train_mask,
                              num_neighbors=[25, 10], shuffle=True, **kwargs)

[Chrisable]

In this way, the nodes of the test set are used during training. Is it still supervised learning?

[rusty1s]

This depends on which strategy you want to use to perform inference (node-wise sampling vs layer-wise sampling). (1) You can use the same strategy as in training (sampling a subgraph for each node). This works just fine, but you will actually re-sample and re-compute a lot of neighbors this way, which is slow (but does not require a dedicated new implemenation). The layer-wise sampling approach (2) works by just sampling the 1-hop neighbor for each node, compute the intermediate node representation via a GNN layer, and then proceed to sample the next-hop. This way, intermediate node representations can be shared and re-used.

[HeVLF]

Hey. Can you share a code for the second approach that you described?

[rusty1s]

https://github.com/pyg-team/pytorch_geometric/blob/master/examples/reddit.py#L51

[HeVLF]

Thank you for your quick reply! Does it make sense to apply this approach (layer-wise evaluation, right?) to a GNN architecture with GINEConv layers that have an MLP as a classifier after?

[HeVLF]

Also, for inductive setting, the following code is the same?
`def inference(self, x_all, subgraph_loader):
pbar = tqdm(total=len(subgraph_loader.dataset) * len(self.convs))
pbar.set_description('Evaluating')

    # Compute representations of nodes layer by layer, using *all*
    # available edges. This leads to faster computation in contrast to
    # immediately computing the final representations of each batch:
    for i, conv in enumerate(self.convs):
        xs = []
        for batch in subgraph_loader:
            x = x_all[batch.n_id.to(x_all.device)].to(device)
            x = conv(x, batch.edge_index.to(device))
            if i < len(self.convs) - 1:
                x = x.relu_()
            xs.append(x[:batch.batch_size].cpu())
            pbar.update(batch.batch_size)
        x_all = torch.cat(xs, dim=0)
    pbar.close()
    return x_all`

[HeVLF]

Our NeighborLoader examples are currently indeed transductive. If you want to perform inductive learning, you can change the loaders as follows:

train_loader = NeighborLoader(data.subgraph(data.train_mask),
                              num_neighbors=[25, 10], shuffle=True, **kwargs)

By doing this there are still test nodes in the data objects generated by the NeighborLoader. They are present in the "n_id" attribute of each data object. I don't think this is suppose to happen.

[rusty1s]

No, they are based on the subgraph. A simple way to fix this is:

mapping = train_mask.nonzero().view(-1)
n_id_on_full_graph = mapping[batch.n_id]

[rusty1s]

Alternatively, you can do

train_mask = ...
sub_data = data.subgraph(train_mask)
sub_data.n_id = train_mask.nonzero().view(-1)

loader = NeighborLoader(sub_data, ...)

for batch in loader:
     print(batch.n_id)  # Batch will respect the `n_id` attribute if it is already set as part of `sub_data`

[HeVLF]

That makes sense, thank you!
And those n_ids are then used for the edge_index tensor?

[rusty1s]

Sorry for late reply. Can you clarify what you mean? n_id[0] just points to the original node index in the first sampled node, and edge_index in sampled subgraph points contains only these local indices (rather than the original n_id[0] index).

[HeVLF]

No problem. Yes, I then understood that. It makes sense. I appreciate your help, I could understand better the NeighborLoader inner workings.