Defining a new abstract node and learning it during training

[sajastu]

Hi,

I’m using Huggingface Transformers to encode a given text, and then initializing the node embeddings of graph with token representations from the Transformers LM. As a part of my application, I need to define a special node (called, for example, [INTERACTION] node) whose embeddings should be learnt during the training. This special node is used as a node that learns how to combine a group of entities in the given graph. I’m a bit lost in this process, though; looking at the documentation, I was not able to find a way to this end. Any ideas?

[wsad1]

One way of doing this would be to add a virtual node that has incoming edges from all other nodes in the graph, as done in this paper, you can find the implementation here.
I am not sure about the best way to implement this, but off the top of my head you need to do two things

x = torch.cat(x, v_node_x) # update feature tensor with virtual node.
edge_index = torch.cat(edge_index, v_node_edges) # add incoming edges to v_node to edge_index.

[sajastu]

@wsad1 Thank you so much for providing the paper + the repo. They were very helpful at implementation. For future reference, I used nn.Embedding(.) for defining the node feature whose weights get updated with each batch of data, meanwhile, I'm modifying the adjacency matrix to add the new connections (i.e., edges) to this virtual node.