Graph Transformer Operator

[rtwod2]

Hello all,

I would like to ask you about the Graph Transformer Operator that you have available. Specifically I' m referring to The graph transformer operator from the “Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification” paper. In this paper, and eventually in your operator, do you make use of positional encodings?

Thank you in advance.

[rusty1s]

The operator itself does not add positional encodings. Since the attention is only performed in local neighborhoods, it is not strictly necessary to use positional encodings here. If you want to add them nonetheless, it is expected that the user provides these as part of the input features, e.g., via transforms.AddLaplacianEigenvectorPE.

[rusty1s]

Unlike in recent Graph Transformer works, which perform attention across all nodes, the TransformerConv performs attention in 1-hop neighborhoods, e.g., it only computes an attention score for every direct neighbor and performs a weighted aggregation. There is no clear meaning of "positional encoding" in local neighborhoods.

[rtwod2]

Hi again,

Thank you for your help. I'm trying to use the AddLaplacianEigenvectorPE as you mentioned above, but I'm a little bit confused. Can you give me a code example on how to use it?

My graph dataset is split into train-validation set, and test set, and then the train-validation set is split into dynamic kfolds so as to use kfold cross-validation. Then I call a function for each one of these sets, and I create the pytorch geomertic graphs. In which step should I use the AddLaplacianEigenvectorPE?
I found this example on git #8081 but it wasn't so helpful on how practically we implement this transform.
Could you please enlighten me with an example?

Thank you very much.

[rusty1s]

This example here shows to use PE transforms: https://github.com/pyg-team/pytorch_geometric/blob/master/examples/graph_gps.py Hope this is useful.

[rtwod2]

Hi again,
It was really helpful, I got an idea of how it works. I have 2 questions:

1. the k in pos_enc = AddLaplacianEigenvectorPE(k=??) should be k<N where N is the minimum number of nodes per batch of graphs?
2. After converting my data.x into node embeddings, and data.edge_attr into edge_embeddings, I have the below dimensions, so as you can see, I have laplacian positional encodings with the dimension [2094,10]
   a. 10 is as I can understand the number of pe_dim, but I choose 10 by chance. I mean I can't understand with which criteria we
   choose it. Can you help me with this?
   b. How can I sum the data.x with the lpe taking into account the problem with the dimensions?

I' m just starting to understand all these, and it's kind of messy on my head :)

Thank you very much in advance

[rusty1s]

1. Yes, k should be usually smaller than the number of nodes in each graph.
2. This is a hyperparameter, and so you can consider tuning it. You can combine it with your node features via x = torch.cat([data.x, data.lpe], dim=-1).