How to train GCN on multiple graphs while the nodes have labels not the total graph

[harirRzi]

Hello. I have multiple graphs and each of its node has features and labels. I am new in Graph neural network. I want to train a GCN with multiple of these graphs while the nodes have labels instead of each graph and then test and evaluate on the rest of graphs which again their nodes have labels. In the test part, I want the GCN predict the label of each nodes when I input any graph with the features of the nodes. I appreciate if any one can help me, how can I do it?

[rusty1s]

This sounds like a node-level prediction task on multiple graphs. examples/ppi.py is similar to that.

[harirRzi]

Thank you for your answer. Now my problem is, sometime the labels of the nodes in each graph are only '0' and sometime the labels contain '0' and '1'. I'm trying to use GCN to train based on mutiple graphs. Here is the code, I have written so far:

class GCN(torch.nn.Module):
    def __init__(self, in_channels, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = GCNConv(in_channels, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, out_channels)

    def forward(self, x,edge_index):
        # x, edge_index = data.x, data.edge_index

        x = self.conv1(x, edge_index)
        x = F.relu(x)
        x = F.dropout(x, training=self.training)
        x = self.conv2(x, edge_index)

        return F.log_softmax(x, dim=1)
        
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
in_channels = gInfo.getNodeNumberOfFeatures()  # Number of input features per node
hidden_channels = gInfo.getNodeNumberOfFeatures()+3 
max_num_classes = 2  # Since the target output can be either 0 or 1

# Initialize the GCN model with the maximum number of classes
model = GCN(in_channels, hidden_channels, max_num_classes).to(device)

# Use Binary Cross Entropy Loss for scenarios where target has only one class
binary_loss_op = torch.nn.BCEWithLogitsLoss()

# Use Cross Entropy Loss for scenarios where target has two classes
multi_loss_op = torch.nn.CrossEntropyLoss()

optimizer = torch.optim.Adam(model.parameters(), lr=0.005)


def train():
    model.train()

    total_loss = 0
    for data in train_loader:
        data = data.to(device)
        optimizer.zero_grad()

        # Forward pass through the model
        model_output = model(data.x, data.edge_index)
        num_nodes=data.x.shape[0]
        # Determine the loss function based on the number of classes in the target
        num_classes = len(torch.unique(data.y))
        if num_classes == 1:
            loss_op = binary_loss_op
            # Ensure that target tensor has 1D shape for Binary Cross Entropy Loss
            target = data.y.float().unsqueeze(dim=1)
        else:
            loss_op = multi_loss_op
            # Ensure that target tensor has 1D shape for Cross Entropy Loss
            target = data.y.squeeze(dim=1)

        # Calculate the loss
        loss = loss_op(model_output, target)

        # Backpropagation and optimization step
        loss.backward()
        optimizer.step()

        total_loss += loss.item() * num_nodes
        total_examples += num_nodes

    return total_loss / total_examples

And after running the project, I get the following error:

File "/Users/anaconda3/envs/GNN/lib/python3.12/site-packages/torch_geometric/nn/conv/gcn_conv.py", line 241, in forward
   edge_index, edge_weight = gcn_norm(  # yapf: disable
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/Users/anaconda3/envs/GNN/lib/python3.12/site-packages/torch_geometric/nn/conv/gcn_conv.py", line 108, in gcn_norm
   deg = scatter(edge_weight, idx, dim=0, dim_size=num_nodes, reduce='sum')
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/Users/anaconda3/envs/GNN/lib/python3.12/site-packages/torch_geometric/utils/_scatter.py", line 75, in scatter
   return src.new_zeros(size).scatter_add_(dim, index, src)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: index 3 is out of bounds for dimension 0 with size 3    

I appreciate, if anyone can suggest me how to fix the error.
My other question is, the labels of the nodes in each graphs are sometime imbalanced. how can I penalize the loss function to make my model to work better on imbalanced data?

[rusty1s]

What happens if you run data.validate() before moving the data to the model?

For your loss question: PyTorch loss functions can take in a weight argument for this use-case.

[harirRzi]

Thank you for your answer. Actually, I input 'weight' to the loss, I get error. So I tried to manually give weights for the loss function. Here is my code, but the prediction is only 1 which is the major class. Could you please check if my code is correct for training by manually create weights for each class? the majority of labels for each of nodes in each of graphs that I input to the GCN has label 1.

class GCN(torch.nn.Module):

    def __init__(self,in_channels, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = GCNConv(in_channels, hidden_channels)
        self.linear = torch.nn.Linear(hidden_channels, out_channels)
    def forward(self, x, edge_index,edge_weight):
        x = self.conv1(x, edge_index,edge_weight)
        x = F.relu(x)
        x = F.dropout(x, training=self.training)
        x = self.linear(x)
        out=x
        out = F.softmax(out, dim=1)
        label = out.argmax(dim=-1)
        return label,out

def train():
    model.train()

    total_loss = 0
    for data in train_loader:
        data = data.to(device)
        data.requires_grad_(True)
        optimizer.zero_grad()
##----------------------------------------------------------------------------------------
        num_classes = len(torch.unique(data.y))
        num_samples_in_class_1 = np.count_nonzero(data.y)
        num_samples_in_class_0 = len(data.y) - num_samples_in_class_1
        total_samples = len(data.y)
        
        if num_samples_in_class_0==0 or num_samples_in_class_1==0:
            # Set weights to 1 if either class has no samples
            weight_for_class_0 = 1
            weight_for_class_1 = 1
        else:
            weight_for_class_0 = total_samples / (num_samples_in_class_0 * num_classes)
            weight_for_class_1 = total_samples / (num_samples_in_class_1 * num_classes)    
##-----------------------------------------------------------------------------------------
        # Convert the weights to a tensor
        weights_tensor = torch.tensor([weight_for_class_0, weight_for_class_1]).to(device)
        label,output = model(data.x, data.edge_index, data.edge_weight)
        # Calculate loss
        loss = criterion(output, data.y)
        weights = torch.where(data.y == 0, torch.tensor(weight_for_class_0), torch.tensor(weight_for_class_1))
        weighted_loss = torch.mean(weights * loss)
        # Calculate accuracy
        acc = accuracy(output.argmax(dim=1), data.y)

        total_loss += weighted_loss.item() * data.num_graphs
        weighted_loss.backward()
        optimizer.step()

    return total_loss / len(train_loader.dataset)

[rusty1s]

you need to do `criterion(output, data.y, weight=weight)

[harirRzi]

here in the method forward() at the end, I want to return the exact class label for each of nodes. the data.y also contains only 0 and 1 as the class of each nodes. Thus I can use BCEWithLogitsLoss() as the loss function. Can you tell me how can I output the predicted class of each of nodes in forward method()?

[rusty1s]

The final class depends on your chosen threshold. For balanced datasets, a threshold of 0.5 can be chosen. Otherwise, you can take the best threshold by looking at precision recall curve.