Multi-class Classification

[anthonysirico]

What shape does my data.y have to be in order to perform multi-class classification? As of right now, data.y.shape = torch.Size([2]), but I keep getting the error

Traceback (most recent call last):
  File "/home/polar/venv/pygeo/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3526, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-18-b6989be4f4a8>", line 1, in <module>
    criterion(out, graphs.y)
  File "/home/polar/venv/pygeo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/polar/venv/pygeo/lib/python3.10/site-packages/torch/nn/modules/loss.py", line 1174, in forward
    return F.cross_entropy(input, target, weight=self.weight,
  File "/home/polar/venv/pygeo/lib/python3.10/site-packages/torch/nn/functional.py", line 3029, in cross_entropy
    return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
ValueError: Expected input batch_size (7) to match target batch_size (14).
graphs.y

So I have a feeling, my data.y shape is wrong. Here is how I have it in the dataset build:

y = [graph_features['compiled'][index], graph_features['simulated'][index]]
pyg.y = torch.tensor(y, dtype=torch.long)

class GCN(torch.nn.Module):
    def __init__(self, hidden_channels):
        super(GCN, self).__init__()
        torch.manual_seed(12345)
        self.conv1 = GraphConv(1, hidden_channels)
        self.conv2 = GraphConv(hidden_channels, hidden_channels)
        self.conv3 = GraphConv(hidden_channels, hidden_channels)
        self.lin = Linear(hidden_channels, 4)

    def forward(self, x, edge_index, batch):
        # 1. Obtain node embeddings
        x = self.conv1(x, edge_index)
        x = x.relu()
        x = self.conv2(x, edge_index)
        x = x.relu()
        x = self.conv3(x, edge_index)

        # 2. Readout layer
        x = global_mean_pool(x, batch)  # [batch_size, hidden_channels]

        # 3. Apply a final classifier
        x = F.dropout(x, p=0.5, training=self.training)
        x = self.lin(x)

        return x

[rusty1s]

The input into F.cross_entropy should be a prediction of shape [batch_size, num_classes], and the target needs to contain indices of shape [batch_size]. How do the shapes look like for you?

[anthonysirico]

@rusty1s So i ended up reshaping it and now it will go through but I wind up with a different error.
The y shape is:

data.y.shape = torch.tensor([1,2])

After putting all 32,000 in the DataLoader and doing:

criterion = F.CrossEntropyLoss
for graphs in training_loader:
    out = model(graph.x.to(torch.float), graphs.edge_index, graphs.batch)
    loss = criterion(out, graphs.y)

Shapes are:

out.shape = torch.tensor([4,4])
graphs.y.shape = torch.tensor([4,2])

The following error:

Traceback (most recent call last):
  File "/home/polar/venv/pygeo/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3526, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-23-1a4a01b1cac6>", line 1, in <module>
    loss(out, graphs.y)
  File "/home/polar/venv/pygeo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/polar/venv/pygeo/lib/python3.10/site-packages/torch/nn/modules/loss.py", line 1174, in forward
    return F.cross_entropy(input, target, weight=self.weight,
  File "/home/polar/venv/pygeo/lib/python3.10/site-packages/torch/nn/functional.py", line 3029, in cross_entropy
    return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
RuntimeError: 0D or 1D target tensor expected, multi-target not supported

From what I read, should use MSELoss since it can handle multiple classes.

[anthonysirico]

I missed labeled my question, its multi-label, not multi-class. I changed the criterion to BCEWithLogitsLoss and the optimizer to SGD.
Each label has a 1 for its associated class and all will have at least 2 ones and 2 zeros, i.e. data.y = torch.tensor([[1.,0.,1.,0.]], dtype=torch.float).

Now i just have to figure out how to measure accuracy.