what are the things to consider while working on a link prediction model

[knitemblazor]

Hi,

i am working on link prediction problem and i am using one of the examples as reference

class Net(torch.nn.Module):
    def __init__(self, in_channels, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = GCNConv(in_channels, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, out_channels)

    def encode(self, x, edge_index):
        x = self.conv1(x, edge_index).relu()
        return self.conv2(x, edge_index)

    def decode(self, z, edge_label_index):
        return (z[edge_label_index[0]] * z[edge_label_index[1]]).sum(dim=-1)

    def decode_all(self, z):
        prob_adj = z @ z.t()
        return (prob_adj > 0).nonzero(as_tuple=False).t()


model = Net(dataset.num_features, 128, 64).to(device)
optimizer = torch.optim.Adam(params=model.parameters(), lr=0.01)
criterion = torch.nn.BCEWithLogitsLoss()


def train():
    model.train()
    optimizer.zero_grad()
    print(dir(train_data))
    exit()
    z = model.encode(train_data.x, train_data.edge_index)

    # We perform a new round of negative sampling for every training epoch:
    neg_edge_index = negative_sampling(
        edge_index=train_data.edge_index, num_nodes=train_data.num_nodes,
        num_neg_samples=train_data.edge_label_index.size(1), method='sparse')

    edge_label_index = torch.cat(
        [train_data.edge_label_index, neg_edge_index],
        dim=-1,
    )
    edge_label = torch.cat([
        train_data.edge_label,
        train_data.edge_label.new_zeros(neg_edge_index.size(1))
    ], dim=0)

    out = model.decode(z, edge_label_index).view(-1)
    # print(out.view(-1).sigmoid())
    # print(edge_label)
    loss = criterion(out, edge_label)
    loss.backward()
    optimizer.step()
    return loss

As this is for a single graph is it correct a use a torch dataloader to serailize and train the same network batchwise by summing the losses and backpropagating them for different graphs. A graph is built from a image hence it cannot be passed all together as a single graph as there is positional info per graph.

Q2. As there is also a provision for passing the edge weights to gcn layer what should be the optimal range of these wights should they be beetween 0-1?

Q3. hwill increasing no.of layers improve the model performance.

Q4. for the decode part is it right to use a mlp and train the pairs as a binaray classifier instead of the pairwise product?

Any suggestion will be greatly appreciated as i have done the same and i am unable to bring down the false positives considerably.

@rusty1s pls help

[EdisonLeeeee]

Hi @knitemblazor maybe I could answer some of your questions:

For Q2: The edge weight does not have to lie between 0 and 1. Actually, it can be any meaningful value but in most cases lying in the range of [0,1] would be better.

For Q3: In literature, the best performance of GCN is achieved when the no. of layers is 2. Increasing it would typically lead to an oversmoothing issue. For the oversmoothing issue, you can refer to this blog

For Q4: You are right. Using MLP as a decoder usually could achieve better performance than a simple pairwise product.

[knitemblazor]

Thank you very much i was very frustrated you gave me hope @EdisonLeeeee