Multilabel Classification

[anthonysirico]

1. Are there any other multilabel classification examples, beside ppi?
2. Why can't I measure things like accuracy when doing multilabel training?

class GCN(torch.nn.Module):
    def __init__(self, hidden_channels):
        super(GCN, self).__init__()
        torch.manual_seed(12345)
        self.conv1 = GraphConv(1, hidden_channels)
        self.conv2 = GraphConv(hidden_channels, hidden_channels)
        self.conv3 = GraphConv(hidden_channels, hidden_channels)
        self.lin = Linear(hidden_channels, 4)

    def forward(self, x, edge_index, batch):
        # 1. Obtain node embeddings
        x = self.conv1(x, edge_index)
        x = x.relu()
        x = self.conv2(x, edge_index)
        x = x.relu()
        x = self.conv3(x, edge_index)

        # 2. Readout layer
        x = global_mean_pool(x, batch)  # [batch_size, hidden_channels]

        # 3. Apply a final classifier
        x = F.dropout(x, p=0.5, training=self.training)
        x = self.lin(x)

        return x

def train(model):
    model.train()

    total_loss = 0
    for graphs in training_loader:
        graphs = graphs.to(device)
        optimizer.zero_grad()
        out = model(graphs.x.to(torch.float), graphs.edge_index, graphs.batch)
        loss = criterion(out, graphs.y)
        total_loss += loss.item() * graphs.num_graphs
        loss.backward()
        optimizer.step()
    return total_loss / len(training_loader.dataset)

[wsad1]

1. Check out ogbn-protiens from https://ogb.stanford.edu/docs/nodeprop/#ogbn-proteins.
2. You could use accuracy for each label. To get an aggregate across all labels you could aggregate the metrics across labels. Read more here

[anthonysirico]

@wsad1 @rusty1s Last question, so after doing the following:

dataset = TMSDataset(path).shuffle()
dataset_copy = dataset.copy()

good_indices = []
bad_indices = []
for i in range(len(dataset)):
    if dataset[i].Utility.item() == 100.:
        bad_indices.append(i)
    else:
        good_indices.append(i)
        
good_set = dataset_copy[good_indices]
bad_set = dataset_copy[bad_indices]

mean = good_set.Utility.mean()
std = good_set.Utility.std()
good_set.Utility = (good_set.Utility - mean) / std
mean, std = mean.item(), std.item()

final_dataset = torch.utils.data.ConcatDataset([good_set, bad_set])

How can I shuffle this dataset up before splitting it into training and validation? Because all the bad stuff is at the end.

[rusty1s]

Mh, good question. That's more of a PyTorch question, and we have our own implementation for that via InMemoryDataset.shuffle(). What you can is either (1) divide good_set and bad_set into individual training/validation samples, and then construct train_dataset and val_dataset out of them, e.g., ConcatDataset([good_train_set, bad_train_set]); (2) Alternatively, you can just shuffle [good_set] + [bad_set] before:

set = good_set + bad_set
set = [set[i] for i in torch.randperm(len(set)).tolist()]