Questions about leaking data

[do-lania]

Hi, I am very new to geometric DL, so I might be misunderstanding something, so any clarifications/pointers are very welcome.

Here's the problem I'm trying to solve: I have one very large graph, where each node has a single value, coordinates and a timestamp. Edges are constructed using coordinates with a RadiusGraph. I am trying to get a GNN to do node-level regression: basically, I want to be able to add a node with random coordinates and have the GNN predict the value.

Here's the way I construct the data: data = Data(x=values, y=values, pos=coords, date=dates, train_mask=train_mask, val_mask=val_mask). Then, I use LightningNodeData to get batches of data (which I believe is based on NeightborSampler).

My questions/concerns are:

• Since my features and labels are the same, I am concerned about leaking data. If I want to predict a value for a node X, which connects to a node Y, wouldn't that mean that after two passes I will get something like X<-Y<-X, leaking the correct answer to the model? If so, how do I avoid this issue?
• I want to ensure that causality is preserved during training, so I am trying to make sure that whenever I predict a value for a node X, I am only using information that was available before the node X was created. I tried to do this by removing edges this way: data.edge_index = data.edge_index[:, data.date[data.edge_index][0] > data.date[data.edge_index][1]]. However, this means the graph is directed now and I wasn't able to find much information on dealing with those. Will this approach work for me?

[rusty1s]

1. Yes, directly using the features will result in data leakage. What you can do is to mask out the features that you want to predict, and only use neighboring node features to infer their values:

def forward(self, batch)
    x = batch.x
    x[:batch.batch_size] = 0.
    # Proceed as usual.

2. With PyG 2.1.0, we now also support temporal sampling (see the NeighborLoader.time_attr attribute for more information).

[do-lania]

I have another question about the first point. When I mask the features I'm trying to predict with 0, if my model has multiple hops of message passing, wouldn't this 0 value basically "pollute" the computation? Is there a way to distinguish between a "node with true 0 value" and a "node with no information available"?

[rusty1s]

That‘s a good point. I think 0 can be safely viewed as „no information“ since matrix multiplication will always produce zero outputs. Alternatively, you can also augment the graph structure in the sense that masked out nodes only receive information, but they cannot distribute information (only one edge direction is maintained).