Node embeddings in a multidirectedgraph (Heterograph with multiple node types and multiple edge types)

[Jeriousman]

Hello PyG team. I want to make sure once and for all not only for me but also someone who would look for clear understanding for hetero node embeddings. I am not entirely sure if my knowledge is right so I wanna ask here.

Let's say I have one big multi-directed graph (Heterograph with multiple node types and multiple edge types) with

Node Types:
User
Movie
Actor

Then we also have 4 edge types as below:

Edge Types:
User-Watched-Movie
User-Liked-Actor
User-Follows-User
Actor-Played-Movie

Then for better embedding update, we add reverse edges as below:

Reverse Edge Types:
Movie-Watched_by-User
Actor-Liked_by-User
User-Followed_by-User
Movie-Played_by-Actor

Now say we do node classification to get node embeddings. Then here is my question:

1. If I want user node embeddings specifically, I still do training/evaluating for all node types and all edge types then I take only user embedding by doing somethingg like return self.lin(x_dict['user'])?

2. If correct, the above 1 operation will be done as said then summed by the same node type in target node? For example, user node embedding would be updated from user-follows-user, movie-watched_by-user, actor-liked_by-user, user-follwed_by-user as user node type is in target node side (So x_dict['user'] will be updated by summing up those?) Or as training is done and all nodes in all node types are updated, I just need to take nodes['user'] simply as that?

3. For hetero node embedding update, do we sum up by node type? or average by node type in general? or both are all wrong?

Another question in the same setting but for link prediction task. I am still interested in node embedding here too. In this case I suppose that we dont have node labels like node classification up there. So we update by edges with negative sampling. In this case,
how I see normally out there for link prediction is (Source node) ⋅ (Target node) ( ⋅ is Dot Product). During this training, we will obtain the updated embeddings of source nodes and target nodes. Here are my questions (similar as above).

4. The source node and target node dot product operation will be done by all edge types if I am correct. Then If I want to get lets say 'user' embeddings, how do I get them? Do I get embeddings of user embeddings by only taking Target Node embedding from (Source node) ⋅ (Target node) when target node is user as target node like in User-Follows-User, Movie-Watched_by-User, Actor-Liked_by-User, User-Followed_by-User? If not, how do we get user node embeddings? Or as training is done and all nodes in all node types are updated, I just need to take nodes['user'] from one of any edge pairs simply as that?

5. For link prediction task in this example, say I want to recommend movie to user. Then should I only take User-Watched-Movie link and see the binary classification? Or should I take Movie-Watched_by-User as well and do something together? (If something, what is the something?)

6. To get the entire user node embeddings for all users in link prediction setting, we train with negative samples and not all nodes are predicted at evaluation time. How can we fetch all of user node embeddings? After all the training is done, should I put full graph as input to the trained model and get the node embeddings? How should we do it? It is very unclear for me.

I guess I am very confused at this moment and maybe I am really not understanding correctly how the hetero node classification and link prediction work. I hope someone can clarify this so we are all clear in the details.

[rusty1s]

1. Yes, this is the suggested approach. In theory, one can reduce the number of node types involved on a per layer basis but it is very troublesome to deal with this.
2. Summing of embeddings will happen for each layer, not only at the end (the HeteroGNN layer will take care of that).
3. You can customize that in all heterogenous GNN variants included in PyG ("sum", "mean", "max", ...).
4. For link prediction, we will only compute dot product for the edge type under consideration (similar to the final readout in your node classification example): x_src = x_dict["user"]; x_dst = x_dst["movie"]; x_src[edge_index[0]] * x_dst[edge_index[1]]. If you also want to return embeddings, you should be able to simply return them inside your model as well.
5. One direction is enough. There is no information you gain by also looking at the reverse direction for the final link-level recommender.
6. During evaluation, it's indeed best to just store node embeddings obtained by your model and apply the link-level predictor afterwards. This way, you save computation by node needing to re-compute embeddings for all links you want to look at.

[Jeriousman]

@rusty1s One small question about your answer 6. For tranductive learning, as you also mentioned, indeed best to store node embeddings at evaluation time. But here, do we use transductive test data? or full data without splits to the trained model? Embeddings with which data would be the final embeddings?

[rusty1s]

What do you mean with transductive test data? In general, you will want to save the node embeddings for all links that either occur as source or destination nodes in your test set.

[Jeriousman]

@rusty1s, I' am sorry if I was not so clear. For example, for transductive settings, the train/valid/test splits for edge_index's size differ. And only to get node embeddings with test dataset will have not too many edges in it. Maybe number of edges in the data would affect node embeddings? Because I experimented to get node embeddings by putting train data, then valid data and then test data independently, I got slightly differnt embeddings (but there were quite similar).

I guess node embeddings are the final x in return x

  x = self.conv1(x, edge_index).relu()
  x = self.conv2(x, edge_index)
  return x  

So I did 3 times of this with train, valid, test data each. As embeddings differs, I just thought maybe full graph data without splits which will hold all edges and x (features). Is it a right way? What would offer the best embedding representation?

So my question is

1. x in [return x] is the final node embeddings we normally call as node embeddings?

2. To get the final best node embeddings, what dataset better be used at evaluation on the trained model? Or I guess node embeddings are not fixed, but it is continuously changing depending on what edge_index data comes in if I wanna get node embeddings for specific dataset, I should put the x and edge_index of that specific dataset? and so on so fort?

3. Maybe you said "During evaluation, it's indeed best to just store node embeddings obtained by your model" because unlike general deep learning models, transductive GNN models are growing size from train < evaluation (valid) < test, we do not save the embeddings of training phase but of evaluation (valid) phase to finally use on test set and future unseen edges?

I wish I got clearer. Thank you in advance!

[rusty1s]

Note that edge_index should be fixed (the edges on which to perform message passing), only the supervision edges (the edges to train against) change across different splits.

1. Yes, that's correct.
2. IMO, node embeddings are fixed.
3. Yeah, the scenario changes a bit for temporal link prediction, in which you want to update node embeddings accordingly as new ground-truth edges come in. Here, it's probably best to not save node embeddings at all, and both do GNN propagation and link-level classification jointly during inference.

Hope this helps!

[Jeriousman]

Thank you very much for your advice!

But edge_index should't be fixed? I think for training/valid/test dataset in disjoint transductive setting, training message passing edges are just training message passing edges, but valid message passing edges are training message passing edges + training supervision edges. Then test set message passing edegs are training message passing edges + training supervision edges + valid supervision edges.
Is it not? So I guess edges_index gets larger?