Link Prediction explanation

[apratim-mishra]

Hi,
are there any libraries or sources on explaining links. (for link prediction task)

I built a model that transforms node embeddings, then predicts links on those embeddings based on new node feat, edge feat, and edge attr;
is there a way I can find which node features or which edge feature are more important (can be for individual edges or for the complete model;

Regards

[rusty1s]

I'm sorry to tell but we currently do not support explainability in link prediction tasks. We would need to bring both GNNExplainer and our newly added to_captum method to support this use-case.

Pinging @RBendias here as well.

[RBendias]

Hi,

I just tried the to_captum method on the pyg example for link prediction. It seems to work, in case you want to know more about node features. However, I have never worked on link prediction and I don't know if the results are reasonable.

The model needs to have a forward function:

class Net(torch.nn.Module):
    def __init__(self, in_channels, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = GCNConv(in_channels, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, out_channels)

    def encode(self, x, edge_index):
        x = self.conv1(x, edge_index).relu()
        return self.conv2(x, edge_index)

    def decode(self, z, edge_label_index):
        return (z[edge_label_index[0]] * z[edge_label_index[1]]).sum(dim=-1)

    def decode_all(self, z):
        prob_adj = z @ z.t()
        return (prob_adj > 0).nonzero(as_tuple=False).t()
    
    def forward(self, x, edge_index, edge_label_index):
        z = self.encode(x, edge_index)
        return self.decode(z, edge_label_index)

model = Net(...)

Explore a particular edge by setting the node_idx argument to the index of the edge you want to examine.

edge_index = 1
captum_model = to_captum(model, mask_type='node', node_idx=edge_index) 
ig = IntegratedGradients(captum_model)
ig_attr_node = ig.attribute(test_data.x.unsqueeze(0),
                              additional_forward_args=(test_data.edge_index, test_data.edge_label_index),
                              internal_batch_size=1)

[RBendias]

Hi,

the name of the variable node_idx is misleading here. to_captum with the use of node_idx is meant for node classification. So a better variable name would be output_idx instead of node_idx. You set the index for the link in the output of your model to be explained. This provides the importance/attribution values for all the features of all the nodes that are used to predict whether or not the link specified by the index is present or not. Does that help with questions 1, 2 & 3?

As for your last question, you can either retrieve the importance for the two embeddings separately ig.attribute(data.embedding_1.unsqueeze(0), additional_forward_args=(data.embedding_2,...)...) or modify your forward function to pass the embeddings directly concatenated.

[rusty1s]

Interesting point @RBendias. We should probably go and re-name that to avoid confusion.

[RBendias]

Yes, I opened a PR #4076.

[apratim-mishra]

Hi,

Thank you, @rusty1s and @RBendias , i was able to get node and edge feature importance for my problem statement;
Summing around the 2 dimensions also gives the overall importance for a particular link in my case.

Just one last question, if I do not input a value for the output_idx and then run the capture-> integrated gradient function, does that give the overall feature importance for all the links .

e.g.
captum_model = to_captum(link_predictor, mask_type='node')
ig = IntegratedGradients(captum_model)
ig_attr_node = ig.attribute(data1.x[data1.edge_index[0]].unsqueeze(0),
additional_forward_args=(data1.x[data1.edge_index[1]].unsqueeze(0), data1.edge_index), internal_batch_size = 1)

gives me a tensor with shape same as data1.x[data1.edge_index[0]].shape ;

is that the feature importance for all the links?
or, is to_captum valid only with the (output_idx specified) / (specific link)

Regards

[RBendias]

Yes, you should get information about the importance of features relative to all links if you don't specify a value for output_idx. This works only if your model output is of the shape [num_links], but not for a classification problem.

[abhishek-quantiphi]

Hi, I've been trying to use to_captum for a link_prediction task. I tried following the example given in this thread as well as in captum_expainability, but I'm getting a KeyError.

This is the GNN model:

class Net(torch.nn.Module):
    def __init__(self, hidden_channels, out_channels, dropout_prob=0.2):
        super().__init__()
        self.conv1 = SAGEConv((-1, -1), hidden_channels)
        self.dropout = Dropout(p=dropout_prob)
        self.conv2 = SAGEConv((-1, -1), out_channels)

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index).relu()
        x = self.dropout(x)
        x = self.conv2(x, edge_index)
        return x

This is the graph structure (heterogeneous graph):

HeteroData(
  skill={
    num_nodes=1752,
    x=[1752, 1536]
  },
  certification={
    num_nodes=39,
    x=[39, 39]
  },
  occ_broad={
    num_nodes=88,
    x=[88, 88]
  },
  occ_major={
    num_nodes=13,
    x=[13, 13]
  },
  occ_minor={
    num_nodes=27,
    x=[27, 27]
  },
  external_skill={
    num_nodes=25291,
    x=[25291, 1536]
  },
  (skill, is_ext_aligned_to, external_skill)={
    edge_index=[2, 1163],
    edge_label_index=[2, 258],
    edge_label=[258]
  },
  (skill, required_by, occ_broad)={ edge_index=[2, 8119] },
  (skill, required_by, occ_major)={ edge_index=[2, 4224] },
  (skill, required_by, occ_minor)={ edge_index=[2, 4220] },
  (skill, enhanced_by, certification)={ edge_index=[2, 496] },
  (external_skill, rev_is_ext_aligned_to, skill)={ edge_index=[2, 1163] },
  (occ_broad, rev_required_by, skill)={ edge_index=[2, 8119] },
  (occ_major, rev_required_by, skill)={ edge_index=[2, 4224] },
  (occ_minor, rev_required_by, skill)={ edge_index=[2, 4220] },
  (certification, rev_enhanced_by, skill)={ edge_index=[2, 496] }
)

This is how I tried using to_captum:

captum_model = to_captum(model, mask_type='edge', output_idx=10)
edge_mask = torch.ones(test_data.num_edges, requires_grad=True, device=device)

ig = IntegratedGradients(captum_model)
ig_attr_edge = ig.attribute(inputs=edge_mask.unsqueeze(0), additional_forward_args=(test_data.x_dict, test_data.edge_index_dict), internal_batch_size=1)

The error message:

[/usr/local/lib/python3.7/dist-packages/torch_geometric/nn/models/explainer.py](https://localhost:8080/#) in set_masks(model, mask, edge_index, apply_sigmoid)
     14               apply_sigmoid: bool = True):
     15     """Apply mask to every graph layer in the model."""
---> 16     loop_mask = edge_index[0] != edge_index[1]
     17 
     18     # Loop over layers and set masks on MessagePassing layers:

KeyError: 0

Is this because the graph is heterogeneous (and it has different ways/functions to access the nodes/edges/indexes)?
Can anyone please me with this? @rusty1s @RBendias

[rusty1s]

We do not have explainer support for heterogeneous graphs at the moment. We only set a single mask at the moment, while heterogeneous graphs would require that we set individual masks per edge type. @RBendias Are you interested in looking into this?

[abhishek-quantiphi]

Okay got it. Thanks for the quick response @rusty1s

[francyya]

We do not have explainer support for heterogeneous graphs at the moment. We only set a single mask at the moment, while heterogeneous graphs would require that we set individual masks per edge type. @RBendias Are you interested in looking into this?

Hi @rusty1s Does pyg support the explainer for Heterograph until now?

[RBendias]

Hi, on master we do support `to_captum` for hetero graphs now. Here you can find instructions on how to use it: https://github.com/pyg-team/pytorch_geometric/blob/master/torch_geometric/nn/models/captum.py#L326 . Hi @rusty1s <https://github.com/rusty1s> Does pyg support the explainer for

…

Heterograph until now?