'RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn' when using optimizers at arbitrary intervals with automatic optimization.

[ethanherron]

Hello, I am getting the following error when trying to override optimizer_step() to execute optimization steps at arbitrary intervals.
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
The code runs without trying to override the optimizer_step() function. And when I do override the optimizer_step() function I have made sure each optimizer used is appropriately accounted for.
For reference I have included the training_step(), configure_optimizers(), and optimizer_step() code for the first two optimizers below. Nonessential code has been replace with comments for clarity in this discussion post.
Any help or ideas are greatly appreciated.
Thanks!

def training_step(self, batch, batch_idx, optimizer_idx):
	'''
	load data
	execute forward pass
	'''
	if optimizer_idx == 0:
		'''
		first optimizer -> adversarial content loss (density) encoder
		'''

		# assign loss value
		content_density_loss = - (content_density_wDistance.mean())

		# dict
		tqdm_dict = {'content_density_loss': content_density_loss}
		self.log_dict(tqdm_dict)
		return content_density_loss

	if optimizer_idx == 1:
		'''
		second optimizer -> adversarial content loss (strain) encoder
		'''

		# assign loss value
		content_strain_loss = - (content_strain_wDistance.mean())

		# dict
		tqdm_dict = {'content_strain_loss': content_strain_loss}
		self.log_dict(tqdm_dict)
		return content_strain_loss

def configure_optimizers(self):
		# initialize learning rates
		gen_LR = 1e-5
		disc_LR = 1e-5

		# optimizers
		opt_idx_0 = torch.optim.Adam(list(self.encoder_content_density.parameters()), lr=gen_LR)

		opt_idx_1 = torch.optim.Adam(list(self.encoder_content_strain.parameters()), lr=gen_LR)
		return [opt_idx_0, opt_idx_1]



def optimizer_step(
		self, epoch, batch_idx, optimizer, optimizer_idx, optimizer_closure,
		on_tpu=False, using_native_amp=False, using_lbfgs=False,
	):
		if optimizer_idx == 0:
			if (batch_idx + 1) % 3 == 0:
				optimizer.step(closure=optimizer_closure)

		if optimizer_idx == 1:
			if (batch_idx + 1) % 3 == 0:
				optimizer.step(closure=optimizer_closure)

[ethanwharris]

Hi @ethanherron I tried to reimplement your problem with our BoringModel but it all worked fine. Usually this error happens if the loss returned by the training step doesn't have gradient. Are you freezing parts of your model or loss between steps? Sorry that I couldn't be of more help. If you're able to reproduce the problem using our BoringModel then we would likely be able to assist you further.

Thanks 😃

[ethanherron]

@ethanwharris thank you for the quick reply. I also failed to replicate the error in the BoringModel. I am not freezing any parts of the model between forward executions or losses. Additionally, sometimes while setting fast_dev_run=True in the Trainer() function I do not get an error, but every time I try and run with a single epoch I will get the error.

Would it be possible to set up an arbitrary optimizer_step scheme with manual optimization?
Thanks!