GNN for node regression task

[zahattar]

I am working on GNN for node regression.

I create my graphs as follows :

nb_of_data_objects = 1000
for i in range(nb_of_data_objects):
# Define edge indices
edge_indices = …..
# Define target values
target_values = …..
# Define node_features
node_features = …..
# Create the Data object
data = Data(x=node_features, edge_index=edge_indices, y=target_values)
data_objects.append(data)
print("Data objects", data_objects)

Then normalise and create data loader

Create a DataLoader for the training/testing data

train_loader = DataLoader(train_data, batch_size=64, shuffle=True)
test_loader = DataLoader(test_data, batch_size=64, shuffle=False)

Is this the correct way to enter the training loop? Does the model takes one graph at a time? 
# Create the GNN model instance
model = GCNModel(1, hidden_size, 1, num_layers)

Define the loss function

criterion = nn.L1Loss(reduction='none')

Define the optimizer

optimizer = optim.Adam(model.parameters(), lr=lr)

Training loop

for epoch in range(num_epochs):
print ("epoch", epoch)
model.train()
running_loss = 0.0
total_nodes = 0
for batch_idx, data_batch in enumerate(train_loader):
optimizer.zero_grad()
for epoch in range(num_epochs):
model.train()
running_loss = 0.0
for batch_idx, data_batch in enumerate(train_loader):

[rusty1s]

Yes, this looks correct. The model will process 64 graphs at once (as defined by batch_size=64).

[zahattar]

Thank you so much for your response. Could you kindly provide further clarification?

While debugging and examining the contents of the train loader, I have encountered uncertainties. Since I am working with a one feature for each node, I have noticed that within the "for loop" (using "batch_idx, data_batch" in enumerate(train_loader)), the dimension of "data_batch" seems to aggregate the dimensions of all features from every graph, concatenated together. Is this interpretation accurate?

Furthermore, I am curious about how the training loop allows us to effectively differentiate and segregate each distinct graph from one another. Is it necessary to incorporate an additional "for loop" in order to iteratively process each individual graph independently?

[rusty1s]

seems to aggregate the dimensions of all features from every graph, concatenated together. Is this interpretation accurate?

Yes, all node features should be concatenated in the first dimension (the node dimension), creating a single "supergraph" holding disjoint subgraphs.

to effectively differentiate and segregate each distinct graph from one another

For node-level predictions, you usually don't need to do anything since the output will be [num_nodes, num_predictions]. For graph-level predictions, take a look at our global pooling operators, e.g., global_mean_pool. The differentiation between different graphs happen through data.batch.

[zahattar]

Thanks again
Could you please explain to me why we don' need to do anything? Is it not crucial for the model, even in node regression tasks, to distinguish between nodes within each graph? This distinction is essential to enable the model to grasp the patterns inherent in the node data of individual graphs. Without this ability to differentiate, effective learning would be compromised.
And in [num_nodes, num_predictions], num_nodes will be batch size* nb of nodes in each graph? num_predictions is for each node?

[rusty1s]

The crucial information here is that any message passing computation only happens between nodes and edges within the same graph, so there are no modifications needed to make this work in a mini-batch scenario. Yes, output is basically [batch_size * num_nodes, num_predictions].