What is the relationship beween accumulate_grad_batches and lr_scheduler?

[yc1999]

I wrote following code:

    def configure_optimizers(self):
        ......
        return [
            {
                'optimizer': optimizer,
                'lr_scheduler': {
                    'scheduler': scheduler,
                    'interval': 'step',
                    'frequency': 1
                }
            }

I choose step as the interval. Actually, I don't understand what step means!!!

In my opinion, step may mean a batch? But when I set Trainer parameter: accumulate_grad_batches=5, will lr_scheduler still execute after one batch or it only execute after every accumulate_grad_batches batches? If the answer is the later, so the step means the call of optimizer.step()?

(I know accumulate_grad_batches can affect optimizer, but I don't know whether it can affect lr_scheduler)

[rohitgr7]

yes, step means optimization step and accumulate_grad_batches will be taken under consideration while calling the lr_scheduler.
Ref code:
https://github.com/PyTorchLightning/pytorch-lightning/blob/8ea39d2c8f68cc33273c3431a310a262e2240cf9/pytorch_lightning/loops/epoch/training_epoch_loop.py#L434-L437

[yc1999]

Thanks : )