Features for GNN nodes

[ilyalasy]

Hey, my problem formulates as follows: I need to choose exactly one node from graph.
In my graph for each of nodes I have one-hot encoded vector of initial features. In particular, graph is constructed from html DOM elements and features are just one-hot encoded tag name. So, for my graph I'll have a lot of similar nodes (same tags) but they will have different edges.
I'm using 2 GATConv layers (one for feature extraction, second for the prediction) and then applying softmax. The idea is to interpret it as a probability to choose particular node (and the desired node should have the biggest probability among other nodes).It turns out that for nodes that have same feature vector (same tags) output probability is also the same, despite the fact they have different edges. After I added some more features, network started working better - now it can learn to get the highest probability for one node.
In my understanding, graph attention should also take edges into the consideration or have I just chosen wrong approach for this task?

[rusty1s]

This depends. GAT utilizes a sparse attention mechanism across local neighborhoods that is solely feature-based, and as such it might not be the best GNN operator to discriminate between different graph structural properties. It's definitely worth it to try out other GNN operators as well, such as GINConv.

[ilyalasy]

I'll definitely try it out, thank you very much for the quick response!

P.S. I know this is very time-consuming to do, but it would be very-very-very nice if torch geometric has some kind of guide regarding how and when use particular graph operation. I think for beginners like me who haven't read tons of papers it can be confusing to choose a valid approach for a particular task. Or maybe I'm missing something from the existing tutorial?
P.P.S. Really great framework, thank you for bringing it to the world! :)

[rusty1s]

Thanks for suggesting this. This is definitely something I might add in the future :)

[ilyalasy]

Actually tried GINConv(Linear), GCNConv, GraphConv, all of them produce same probability for each node with same features. Are there any more advices?

[rusty1s]

Mh, if specific nodes have the same probability than that means that their learned node representations are equal/similar, and that means that subgraphs around equal nodes are similar as well. I don't have a really good advice for you (this is the natural inductive bias that we aim for). One idea you might wanna try is to add random node feature values to each node feature vector, see https://arxiv.org/abs/2002.03155. That way, the GNN gets more expressive (it can learn from which node a specific information is coming from).

[ilyalasy]

I see, in my case subgraphs are indeed similar. Thanks for the link, appreciate it!