Using output of NN as edge_weights raises backward gradient runtime error inplace operation

[Mandeep-Rathee]

We want to use the edge_weight parameter of SGC (and later also other GNNs). However, if we used a neural network to compute the edge_weight, we get an error in the backward call of the second epoch. The first epoch does not throw an error. Since our first idea was that our own code for the neural network caused the error, we replaced this by GAT or to be more precise the attention weights retrieved by GAT. However, the error still remains the same. Can you help us solving this problem?

We also tried using the __explain_mask__ and __explain__ parameters (ignoring the sigmoid for a moment), but sadly it gives results in the same issue.

We checked it with the following version:

Python 3.6.7, torch 1.6.0, pytorch-geometric 1.6.1

Minimal Example

import torch 
import torch.nn.functional as F 
from torch_geometric.datasets import Planetoid 
from torch_geometric.nn import SGConv 
from torch_geometric.nn.conv.gat_conv import GATConv 
 
dataset = Planetoid("../tmp/PubMed", "Pubmed") 
data = dataset[0] 
 
 
class ExampleNet(torch.nn.Module): 
    def __init__(self): 
        super(ExampleNet, self).__init__() 
        self.conv1 = SGConv( 
            dataset.num_features, dataset.num_classes, K=2, cached=True) 
        self.GAT = GATConv(dataset.num_features, 16, add_self_loops=False) 
 
    def forward(self): 
        x, edge_index = data.x, data.edge_index 
        _, (_, attention) = self.GAT(x, edge_index, return_attention_weights=True) 
        x = self.conv1(x, edge_index, edge_weight=attention.squeeze()) 
        return F.log_softmax(x, dim=1) 
 
 
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') 
model, data = ExampleNet().to(device), data.to(device) 
optimizer = torch.optim.Adam(model.parameters(), lr=0.2, weight_decay=0.005) 
 
best_val_acc = test_acc = 0 
model.train() 
with torch.autograd.set_detect_anomaly(False): 
    for epoch in range(1, 101): 
        optimizer.zero_grad() 
        out = model()[data.train_mask] 
        loss = F.nll_loss(out, data.y[data.train_mask]) 
        loss.backward(retain_graph=True) 
        optimizer.step() 
        print("epoch") 

Error

 
File "sgc.py", line 53, in <module> 
train() 
File "sgc.py", line 37, in train 
F.nll_loss(model()[data.train_mask], data.y[data.train_mask]).backward(retain_graph= True) 
File "(…)/torch/tensor.py", line 185, in backward 
torch.autograd.backward(self, gradient, retain_graph, create_graph) 
File "(…)/torch/autograd/__init__.py", line 125, in backward 
Variable._execution_engine.run_backward( 
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1, 32]] is at version 1; exp 
ected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True). 

We are able to execute the code with torch 1.4 and pytorch-geometric 1.6.1, but we guess that it was not caught. Inplace change detection was extended in torch 1.5, if we have searched correctly. So it is simply not detected in torch 1.4, but probably the gradients are wrong.

[rusty1s]

Removing cached=True fixes this issue.

[Mandeep-Rathee]

It solved my issue.

thanks a lot.