Performing multiple optimizer steps for each training step?

[stwerner97]

Hi, I am working with sequence data of considerable lengths and use truncated backpropagation through time to train an LSTM model. Given the large sequence lengths, I want to perform intermediate updates to the network. So far, training steps process batches of sequences split into TBTT chunks.

Is it possible to perform multiple gradient updates in training_step?

[akihironitta]

Here's the doc page: https://pytorch-lightning.readthedocs.io/en/1.7.7/guides/data.html?highlight=tbptt#truncated-backpropagation-through-time-tbptt