Node masking in GVAE

[shrimonmuke0202]

Hi,

I was trying to implement GVAE and there I found that negative sampling is being applied for edge_index only, is there any way to apply similar kind of techniques for nodes also during reconstruction to increase the robustness of the model performance?

[rusty1s]

I think you can apply dropout or add random gaussian noise to your features to increase robustness. WDYT?

[shrimonmuke0202]

Thanks for your quick response,

Is this right way to implement GVAE with random noise during reconstruction, Please find the demo code below

class VGAEEdgeEmbedding(nn.Module):
  def __init__(self):
    super().__init__()
    self.gcn_encoder = GCNConv(5,5) 
    self.VGAE = VGAE(encoder=self.gcn_encoder)
    # self.GatConv = GATConv(in_channels=32,out_channels=64,heads=2,edge_dim=32)
    # self.mlp_edge = Linear(in_channels=2*32,out_channels=32)
    random_noise = torch.nn.Parameter(torch.Tensor(5,5),requires_grad=True)



  def forward(self,data):
    z = self.VGAE.encode(data)
    z_ = z + random_noise

    # h_src = z[data.edge_index[0]]
    # h_dst = z[data.edge_index[1]]


    return z

[rusty1s]

I don't think you want to make this noise trainable - it will just learn to map it to zero. In addition, I think you would need to apply this before encoding:

x = data.x + 0.1 * torch.randn_like(x)
z = VGAE.encode(x, data.edge_index)

[shrimonmuke0202]

Thanks for your response,
I have another query,
I have a graph that contains isolated nodes, when I apply negative sampling on the edge index I get different outputs when I run the same code multiple times. How can I get rid of these types of problems?

edge_index=` torch.tensor([[0, 1, 1, 2],
                           [1, 0, 2, 1]], dtype=torch.long)
x = torch.tensor([[-1], [0], [1],[6]], dtype=torch.float)

neg_edges = negative_sampling(data.edge_index,num_nodes=4,force_undirected=False,method="dense")

output1 tensor([[3, 0, 2, 1],
        [1, 2, 0, 3]])

output2 tensor([[1, 3, 2, 0],
        [3, 1, 0, 3]])

Another query is how can I construct dual of a graph?

[rusty1s]

Try

from torch_geometric import seed_everything

seed_everything(12345)

[shrimonmuke0202]

Thanks for your answer,
Actually, I want to add all edges between isolated nodes and the other nodes using negative_sampling()

For example,
in the above example node, 3 is an isolated node,
(0,3),(3,0),(1,3),(3,1),(2,3) and (3,2) in the output of the negative_sampling() function.

[rusty1s]

Can't you simply do something like

row = torch.full((data.num_nodes), fill_value=3, dtype=torch.long)
col = torch.arange(data.num_nodes)
edge_index = torch.stack([row, col], dim=0)
edge_index = remove_self_loops(edge_index)

[shrimonmuke0202]

Is negative sampling edges for isolated nodes necessary for VGAE training?

[rusty1s]

Well, negative sampling will sample a subset of all negative edges. As such, it may sometimes sample negatives of isolated nodes and sometimes not. I would say the fact that there exists isolated nodes isn't worth special treatment here.

[shrimonmuke0202]

How to handle the error "No negative edges to sample" in batched negative sampling?

[rusty1s]

If this is the case than you are working on a fully-connected graph (there exists an edge between every pair of nodes). In this case, negative sampling should be skipped.