Decoding Latent Variables into Edge Probabilities for a given Node Pairs

[spyderweb-abdul]

I refer to this implementation:

def forward(self, z, edge_index, sigmoid=True):
        r"""Decodes the latent variables :obj:`z` into edge probabilities for
        the given node-pairs :obj:`edge_index`.

        Args:
            z (Tensor): The latent space :math:`\mathbf{Z}`.
            sigmoid (bool, optional): If set to :obj:`False`, does not apply
                the logistic sigmoid function to the output.
                (default: :obj:`True`)
        """
        value = (z[edge_index[0]] * z[edge_index[1]]).sum(dim=1)
        return torch.sigmoid(value) if sigmoid else value

in this page: https://pytorch-geometric.readthedocs.io/en/latest/_modules/torch_geometric/nn/models/autoencoder.html#InnerProductDecoder

How would the decoding look like in case of a multi-dimensional edge weights? I mean, would there be need for sum() in:
value = (z[edge_index[0]] * z[edge_index[1]]).sum(dim=1)

or, we just multiply the latent variables alone?

[wsad1]

A GCN encoder followed by an InnperProductDecoder is used to learn node embeddings on the link prediction task as described in this paper. This is built to predict a single link between nodes.

If my understanding is correct in your task there are multiple kinds of links between nodes. For that i guess you could build a separate encoder decoder model for each kind of link. Or you could have one encoder which feeds into multiple parametrized decoders. If your goal is to predict an edge weight, instead of presence or absence of edges, you'd have to change both operations in the decoder function.

Also the sum term will always be present. It's present to sum up the dimensions after element wise multiplication of the latent vector.

[wsad1]

So, i couldn't find any paper for link weight prediction. But just off the top off my head you could try something like this.

def forward(self, z, edge_index, sigmoid=True):
        r"""Decodes the latent variables :obj:`z` into edge probabilities for
        the given node-pairs :obj:`edge_index`.

        Args:
            z (Tensor): The latent space :math:`\mathbf{Z}`.
            sigmoid (bool, optional): If set to :obj:`False`, does not apply
                the logistic sigmoid function to the output.
                (default: :obj:`True`)
        """
        value = (z[edge_index[0]] * z[edge_index[1]]).sum(dim=1)
        link_probability=torch.sigmoid(value) if sigmoid else value
        
        #link_weight=0 if link not present else its func(z0,z1)
        #func could be any function. it could be  ||z0-z1||. or maybe a fully connected network.
        link_weight=torch.where(link_probability>0.5,
                              func(z[edge_index[0]],z[edge_index[1]]),0)

        return link_weight

Note this is just an initial thought. So you'll have to try out a bunch of experiments, or better find a good paper that tackles link weight prediction.

[wsad1]

Just found this paper . They tackle the temporal link weight prediction problem. So the discussions about the sparsity of A(adjacency matrix) and how to design a loss function might be relevant to you.

[spyderweb-abdul]

Thank you @wsad1 for this direction. I will try this with the hope that I can succeed. I also tried the following already, even though the error loss result is not favourable yet:

class Decoder(nn.Module):
    def __init__(self, in_feat, out_feat, act=torch.sigmoid, dropout=0., bias=True):
        super(Decoder, self).__init__()        

        self.act = act
        self.dropout = dropout
        self.in_feat = in_feat
        self.out_feat = out_feat
        self.bias = bias     
        self.edge_nn = nn.Linear(self.in_feat, self.out_feat, self.bias)

    def forward(self, z, edge_index):
        z = F.dropout(z, self.dropout, training=self.training)          
        x = torch.cat((z[edge_index[0]] , z[edge_index[1]]), 1)
        x = self.act(self.edge_nn(x))  
        return x

Do you think this is okay too, maybe?

[wsad1]

@spyderweb-abdul , yeah this looks ok. its a really interesting problem, so good luck with your experiments.

[spyderweb-abdul]

Thank you @wsad1!