My GCN model isn't able to predict on test data after training it

[harirRzi]

Hello. I have multiple graphs with different sizes of nodes. I am using the nodes features and for edge features, I am trying to use euclidean distance between each pair of node features. sometimes the labels for nodes with label 1 are higher than the nodes with labels 0 and sometimes 0s are higher than 1 in each of graphs, even sometimes a graph nodes has only labels 0. I tried to train a GCN model using the node features and edge features of 80 percent of the graphs in the dataset. after running the trained GCN on the test part which is on 10 percent of the graphs, here is what I see in the output:

Epoch: 001, Loss: 156.8599, Val: 0.0000, Test: 0.0000
Epoch: 002, Loss: 2.4462, Val: 0.0000, Test: 0.0000
Epoch: 003, Loss: 1.5255, Val: 0.0000, Test: 0.0000
Epoch: 004, Loss: 1.1512, Val: 0.0000, Test: 0.0000
Epoch: 005, Loss: 1.1482, Val: 0.0000, Test: 0.0000
Epoch: 006, Loss: 0.9464, Val: 0.0000, Test: 0.0000
Epoch: 007, Loss: 0.7014, Val: 0.0000, Test: 0.0000
Epoch: 008, Loss: 0.6636, Val: 0.0000, Test: 0.0000
Epoch: 009, Loss: 0.5627, Val: 0.0000, Test: 0.0000
Epoch: 010, Loss: 0.5583, Val: 0.0000, Test: 0.0000
Epoch: 011, Loss: 0.5313, Val: 0.0000, Test: 0.0000
Epoch: 012, Loss: 0.5248, Val: 0.0000, Test: 0.0000
Epoch: 013, Loss: 0.4925, Val: 0.0000, Test: 0.0000
Epoch: 014, Loss: 0.5026, Val: 0.0000, Test: 0.0000
Epoch: 015, Loss: 0.4914, Val: 0.0000, Test: 0.0000
Epoch: 016, Loss: 0.4926, Val: 0.0000, Test: 0.0000
Epoch: 017, Loss: 0.4916, Val: 0.0000, Test: 0.0000
Epoch: 018, Loss: 0.4901, Val: 0.0000, Test: 0.0000
Epoch: 019, Loss: 0.4849, Val: 0.0000, Test: 0.0000
Epoch: 020, Loss: 0.4822, Val: 0.0000, Test: 0.0000
Epoch: 021, Loss: 0.4801, Val: 0.0000, Test: 0.0000
Epoch: 022, Loss: 0.4838, Val: 0.0000, Test: 0.0000
Epoch: 023, Loss: 0.4814, Val: 0.0000, Test: 0.0000
Epoch: 024, Loss: 0.4818, Val: 0.0000, Test: 0.0000
Epoch: 025, Loss: 0.4772, Val: 0.0000, Test: 0.0000
.....
Epoch: 100, Loss: 0.4795, Val: 0.0000, Test: 0.0000

and here is my code:

class GCN(torch.nn.Module):
    def __init__(self, in_channels, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = GCNConv(in_channels, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, out_channels)

    def forward(self, x,edge_index,edge_weight):
        # x, edge_index = data.x, data.edge_index

        x = self.conv1(x, edge_index,edge_weight)
        x = F.relu(x)
        x = F.dropout(x, training=self.training)
        x = self.conv2(x, edge_index, edge_weight)

        return F.log_softmax(x, dim=1)

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
in_channels = gInfo.getNodeNumberOfFeatures()  # Number of input features per node
hidden_channels = gInfo.getNodeNumberOfFeatures()+4  # Number of output features for the first convolutional layer
max_num_classes = 2  # Since the target output can be either 0 or 1
# Initialize the GCN model with the maximum number of classes
model = GCN(in_channels, hidden_channels, max_num_classes).to(device)
# Use Binary Cross Entropy Loss for scenarios where target has only one class
binary_loss_op = torch.nn.BCEWithLogitsLoss()

# Use Cross Entropy Loss for scenarios where target has two classes
multi_loss_op = torch.nn.CrossEntropyLoss()

optimizer = torch.optim.Adam(model.parameters(), lr=0.005)

def train():
    model.train()
 total_loss = 0
    total_examples = 0

    for data in train_loader:
        data = data.to(device)
        optimizer.zero_grad()
        num_samples_in_class_1 = np.count_nonzero(data.y)
        num_samples_in_class_0 = len(data.y) - num_samples_in_class_1
        total_samples = len(data.y)

        # Calculate weights only if both classes have samples
        if num_samples_in_class_0 > 0 and num_samples_in_class_1 > 0 and num_samples_in_class_1 < num_samples_in_class_0:
            weight_for_class_0 = total_samples / (num_samples_in_class_0 * 2)  # Multiply by 2 since there are 2 classes
            weight_for_class_1 = total_samples / (num_samples_in_class_1 * 2)  # Multiply by 2 since there are 2 classes
        else:
            # Set weights to 1 if either class has no samples
            weight_for_class_0 = 1
            weight_for_class_1 = 1
        model_output = model(data.x, data.edge_index, data.edge_weight)
        num_nodes = data.x.shape[0]

        # Determine the loss function based on the number of classes in the target
        num_classes = len(torch.unique(data.y))
        if num_classes == 1:
            loss_op = binary_loss_op
            # Ensure that target tensor has 2 channels for Binary Cross Entropy Loss
            target = torch.zeros(data.y.size(0), 2, device=device)
            target[torch.arange(data.y.size(0)), data.y.view(-1)] = 1
            # Calculate the loss with manually applied weights
            weight = torch.tensor([weight_for_class_0, weight_for_class_1], device=device)
            loss = loss_op(model_output, target) * weight
        else:
            loss_op = multi_loss_op
            # Ensure that target tensor has 2 channels for Cross Entropy Loss
            target = F.one_hot(data.y.squeeze(dim=1), num_classes=2).float()
            # Calculate the loss with manually applied weights
            class_weights = torch.tensor([weight_for_class_0, weight_for_class_1], device=device)
            loss = loss_op(model_output, target) * class_weights
        mean_loss = loss.mean()

        # Backpropagation and optimization step
        mean_loss.backward()
        optimizer.step()

        total_loss += mean_loss.item() * num_nodes
        total_examples += num_nodes

    return total_loss / total_examples
    
    @torch.no_grad()
def test(loader):
    model.eval()

    ys, preds = [], []
    for data in loader:
        ys.append(data.y)
        out = model(data.x.to(device), data.edge_index.to(device),data.edge_weight.to(device))
        preds.append((out > 0).float().cpu())

    y, pred = torch.cat(ys, dim=0).numpy(), torch.cat(preds, dim=0).numpy()
    return f1_score(y, pred, average='micro') if pred.sum() > 0 else 0


times = []
for epoch in range(1, 101):
    start = time.time()
    loss = train()
    val_f1 = test(val_loader)
    test_f1 = test(test_loader)
    print(f'Epoch: {epoch:03d}, Loss: {loss:.4f}, Val: {val_f1:.4f}, '
          f'Test: {test_f1:.4f}')
    times.append(time.time() - start)
print(f"Median time per epoch: {torch.tensor(times).median():.4f}s")

I am wondering if my code has a problem so it can not predict well at all. I appreciate if anyone can suggest me a solution regarding the issue. Also do you have suggestion to use other types of GNN instead of GCN to have better result in this situation?

[rusty1s]

Note that you shouldn't apply log_softmax if you are using CrossEntropyLoss, see https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html

[rusty1s]

You can still apply a final softmax if you are interested in probabilities. However, you shouldn't use the softmax if you plan to use CrossEntropyLoss since this applies the softmax internally already.

[harirRzi]

Thank you for your answer. Yes I want to use cross entropy as the loss function so what is your suggestion to use instead of siftmax to get the exact predicted label instead of probability?

[rusty1s]

If you want the label, then you can just do out.argmax(dim=-1).

[harirRzi]

Thank you for your answer. Actually, I changed my forward() method like the following:

    def forward(self, x,edge_index,edge_weight):

        x = self.conv1(x, edge_index,edge_weight)
        x = F.relu(x)
        x = F.dropout(x, training=self.training)
        x = self.conv2(x, edge_index, edge_weight)
        # Apply the ReLU activation function
        x = F.relu(x)
        # Apply the linear layer
        x = self.linear(x)
        # return F.log_softmax(x, dim=1)
        out=x
        return out.argmax(dim=-1)

I want to know if it produces a correct output? because as far as I know, the line, x = self.linear(x) produces node embedding and then if I return out.argmax(dim=-1) then is it the correct predicted label? Also is it correct if I use nn.BCEWithLogitsLoss() as the loss function because the groundtruth labels are 0 or 1? Or it's better to use torch.nn.CrossEntropyLoss?

[rusty1s]

That produces a predicted label, but you will lose the gradients. You should return out to input into your loss, and out.argmax(dim=1) to get the labels. If you are working with binary classification, BCEWithLogitsLoss can (and probably should) be used.