RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [256, 1]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead.

[dilshan-dev]

Hello,

I am getting the following error:

... saving models ...
episode 0 score 17.0 traiiling 100 games average 17.0 steps 17 InvertedPendulumBulletEnv-v0 scale 2
episode 1 score 17.0 traiiling 100 games average 17.0 steps 34 InvertedPendulumBulletEnv-v0 scale 2
episode 2 score 17.0 traiiling 100 games average 17.0 steps 51 InvertedPendulumBulletEnv-v0 scale 2
... saving models ...
episode 3 score 20.0 traiiling 100 games average 17.8 steps 71 InvertedPendulumBulletEnv-v0 scale 2
... saving models ...
episode 4 score 37.0 traiiling 100 games average 21.6 steps 108 InvertedPendulumBulletEnv-v0 scale 2
episode 5 score 14.0 traiiling 100 games average 20.3 steps 122 InvertedPendulumBulletEnv-v0 scale 2
episode 6 score 29.0 traiiling 100 games average 21.6 steps 151 InvertedPendulumBulletEnv-v0 scale 2
... saving models ...
episode 7 score 36.0 traiiling 100 games average 23.4 steps 187 InvertedPendulumBulletEnv-v0 scale 2
episode 8 score 14.0 traiiling 100 games average 22.3 steps 201 InvertedPendulumBulletEnv-v0 scale 2
episode 9 score 16.0 traiiling 100 games average 21.7 steps 217 InvertedPendulumBulletEnv-v0 scale 2
/home/acar/anaconda3/envs/mysac/lib/python3.7/site-packages/torch/autograd/init.py:199: UserWarning: Error detected in AddmmBackward0. Traceback of forward call that caused the error:
File "main.py", line 40, in
agent.learn()
File "/home/acar/Documents/mysac/sac.py", line 100, in learn
action, log_probs = self.actor.sample_normal(state, reparameterize=True)
File "/home/acar/Documents/mysac/networks.py", line 83, in sample_normal
mu, sigma = self.forward(state)
File "/home/acar/Documents/mysac/networks.py", line 77, in forward
sigma = self.sigma(prob.clone())
File "/home/acar/anaconda3/envs/mysac/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/acar/anaconda3/envs/mysac/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
File "/home/acar/anaconda3/envs/mysac/lib/python3.7/site-packages/torch/fx/traceback.py", line 57, in format_stack
return traceback.format_stack()
(Triggered internally at ../torch/csrc/autograd/python_anomaly_mode.cpp:114.)
allow_unreachable=True, accumulate_grad=True) # Calls into the C++ engine to run the backward pass
Traceback (most recent call last):
File "main.py", line 40, in
agent.learn()
File "/home/acar/Documents/mysac/sac.py", line 122, in learn
critic_loss.backward()
File "/home/acar/anaconda3/envs/mysac/lib/python3.7/site-packages/torch/_tensor.py", line 489, in backward
self, gradient, retain_graph, create_graph, inputs=inputs
File "/home/acar/anaconda3/envs/mysac/lib/python3.7/site-packages/torch/autograd/init.py", line 199, in backward
allow_unreachable=True, accumulate_grad=True) # Calls into the C++ engine to run the backward pass
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [256, 1]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

I tried sigma = self.sigma(prob.clone()) as a possible fix, but the same error was there when it was sigma = self.sigma(prob).

Any help that can be provided is highly appreciated.
Thank you!