Attention Weights

[sincyvt]

Is it possible to provide edge weights/ node weights as attention scores (α_ij between nodes i and j)directly, instead of calculatiing attention scores

[rusty1s]

In that case, you can leverage GraphConv and simply input an additional edge_weight vector to your layer.

[sincyvt]

Thankyou so much for answering my questions. While testing GENCONV, each time the result varies. What may be the expected reason

[rusty1s]

Yeah, GCN only supports single edge weights, so you could use any GNN layer that supports multi-dimensional edge features. Can you clarify what you mean with varying results? Do you fix the seed? Other than that, a small variance is expected since reductions are non-deterministic since the order of floating point operations is undetermined.

[sincyvt]

Thankyou so much for answering my questions. While testing GENCONV, each time the result varies. What may be the expected reason

Solved. thankyou.

[sincyvt]

GCNConv, GENConv, TransformerConv these all are convolutional layers. Will you please explain the need of a convolutional layer? How it affect a model? Can we define different models by simply changing this convolutional layers only

[rusty1s]

Yes, these are different GNN layers/message passing layers proposed in literature, which different strengths and properties, e.g., making use of attention when aggregating neighborhood information.

[sincyvt]

"import torch_geometric.data" produces the following error
OSError: /home/hp2/anaconda3/envs/myphd/lib/python3.7/site-packages/torch_sparse/_diag_cuda.so: undefined symbol: _ZN3c106detail19maybe_wrap_dim_slowEllb
Any suggestions to solve it

[rusty1s]

This suggests that your torch-sparse version is incompatible with your installed PyTorch version.

[sincyvt]

How to resolve it?

[rusty1s]

Depends on how you installed torch-sparse. I suggest to run pip uninstall torch-sparse repeatedly, and then try to re-install it by ensuring you install the right package according to torch.__version__ and torch.version.cuda.

[sincyvt]

Sir,
Is there any Language Model applications like BERT to be combined with GNN. i.e, whether it is any GNN models directly available for natural language processing(NLP)

[akihironitta]

I'd suggest creating another discussion to get more visibility from the community as your question doesn't seem to be relevant to this discussion. (I'd recommend RelBench where they combine tabular data (that can potentially have textual columns) with GNNs. https://github.com/snap-stanford/relbench/blob/45b9d782d44deafeb08d517bfc5ac4e72b4a0eec/examples/gnn.py)