How to accumulate grad batches for GANs in pytorch_lightning >= 1.3.3?

[GuillaumeRochette]

Hi,

Since the 1.3.3, I cannot use the flag accumulate_grad_batches for the training of GANs.
It is immediately reproducible on the GAN example given in pl_examples.domain_templates.generative_adversarial_net.

For any version higher than 1.3.2, in the Trainer, if accumulate_grad_batches is set to any value higher than 1, after one batch (batch_idx == 1) the following error arises:

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

This is due to the fact that the g_loss does not require gradients, because the generator's output does not requires any gradients.

Is it intended?
If yes, what is the workaround to train GANs while still accumulating gradients?

Thanks in advance,

Guillaume

[awaelchli]

Hey, I believe this is the same bug I fixed here: #8284
Will try to put it into the next bugfix release.

As a workaround, you can try to manually call self.untoggle_optmizer(optimizer) in your LM.

[GuillaumeRochette]

Awesome, any idea on when it'll be released in the pip package?

[awaelchli]

sometime this week, no promises :)