Add positional embeddings to GATConv

[Sanjaykrishnamurthy]

How can we add positional embedding to node embeddings in GATConv layers like in transformers?
Is it ok to pass it as edge_attr instead?

[EdisonLeeeee]

If I understood it correctly, the positional embeddings are node-related, so you can add them to the node feature matrix x before passing them to GATConv.

[Sanjaykrishnamurthy]

Thanks for your reply. One thing to note, though, is in my particular case, the nodes are repeated, say for instance, x0->x1->x0-x2, etc. In this context, data.x represents the unique values {x0, x1, x2}, and the edge_index is represented as torch.tensor([[0,1,0],[1,0,2]]). As you can observe, x0 appears twice. Ideally, I want to achieve something like this: x0+pos_emb(0), x1+pos_emb(1), x0+pos_emb(2), and x2+pos_emb(3)

[EdisonLeeeee]

from torch_geometric.utils import scatter

position = torch.tensor([0, 1, 2, 0]) # x0->x1->x2-x0
pos_emb = torch.tensor([[1, 1, 1, 1],
                        [2, 2, 2, 2],
                        [3, 3, 3, 3],
                        [4, 4, 4, 4]]) # position embedding
x = torch.ones(3, 4) # node feature matrix

x = scatter(pos_emb, position, dim=0) + x # position embedding + node feature matrix

Does that work for you?

[Sanjaykrishnamurthy]

This isn't exactly what I wanted. Here the positions of x at 0 and 4 are added together. whereas I want x at 0 to add withed with pos_emb[0] and x at 4 to be added with pos_emb[4].

[EdisonLeeeee]

But you have only 3 nodes here. What does "x 4" stand for?