How to choose 'epochs' parameter for the GNNExplainer?

[urmeya]

When using GNNExplainer as explanation algorithm, what is the basis on which to choose the number of epochs parameters?

I have trained a graph model for binary node classification task. I am using node_mask parameter in explainer as 'attributes' (every node has a feature vector of dim 64) and edge_mask parameter as 'object'.
I tried using different values of epochs.
As I increase epochs, the total sum of node features attributions matrix keeps on decreasing. (E.g.~70000 for 200 epochs and ~5 for 50000 epochs)
If I check the relative importance of different nodes for different epoch setting, self node always gets max importance compared to other nodes but for smaller epochs e.g. 200 it is only 0.03% and after more epochs e.g. 50000 it is 65%.

So overall it seems explanation changes hugely based on choice of number of epochs.
Is there any thereorical or practical criteria to choose appropriate number of epochs for GNNExplainer?

[rusty1s]

Copy over from Slack:

This is a very good question for which I don't have a good answer. GNNExplainer basically works via multiple loss function (decrease attribution in general but keep prediction close to the initial prediction). If you are seeing attribution still heavily decreasing after many epochs, this might be a signal that this multi-task loss is not properly weighted.
You can think also think about adding early stopping and evaluation in explainers in general, and we started to integrate this via explainability metrics (https://pytorch-geometric.readthedocs.io/en/latest/modules/explain.html#explanation-metrics).

[sumone-compbio]

@rusty1s thank you, but can you save the GNNExplainer results with something like random seeds to reproduce your results?

[sumone-compbio]

Never mind please, I figured it out. Thank you so much once again.

[sumone-compbio]

Hi @rusty1s, can you review my code to get explanations for multiple graphs:

`

explainer = Explainer(
    model=model,
    algorithm=GNNExplainer(lr=0.0005, epochs=4000),
    explanation_type='model',
    node_mask_type='attributes',
    edge_mask_type='object',
    model_config=dict(
        mode='binary_classification',
        task_level='graph',
        return_type='raw',
    ),
)

def set_seed(seed):
    torch.manual_seed(seed)
    torch.cuda.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)  # for multi-GPU.
    np.random.seed(seed)
    random.seed(seed)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False

 seed = 42
 all_explanations = []
 unfaith_metrics = []

 with torch.no_grad():
     with torch.set_grad_enabled(True):
         for data in test_loader:
       
             set_seed(seed)
        
             X = data.x.float()
             Edge_index = data.edge_index.long()
             Batch_index = data.batch.long()
    
             explanation = explainer(x=X, edge_index=Edge_index, batch_index=Batch_index)
             metric = unfaithfulness(explainer, explanation)
    
             all_explanations.append(explanation)
             unfaith_metrics.append(metric)

          test_all_graphs = Batch.from_data_list(all_explanations)

`

I want my explainer model to be optimized for all my input graphs. Please suggest any changes that would be helpful. Thank you!

[sumone-compbio]

Sorry, I just checked that GNNExplainer is an instance-based method, not a model-based one. So the learning rate and epochs may differ from graph to graph, right?

[rusty1s]

That is correct, although I assume that you should be able to share LR and #epochs without much performance loss.