How to implement an iterative optimization per data reusing the previous prediction for the next iteration

[geogeo28]

Hi,

I am working on a ML problem which requires an optimization of n iterations of a model using the previous prediction such as:
noise → forward → backward → forward ->backward…
this is done per image with a batch of one on 1 GPU.
the image/data is only use in the loss.

What will be the simplest and best way to implement this kind of loop with Pytorch Lightning? Create my own loop? A custom data loader that loads an image per epoch and use the training_steps as optimization iterations or implement my own manual optimizer?

This is similar to the Deep Image Prior work GitHub - DmitryUlyanov/deep-image-prior: Image restoration with neural networks but without learning. but using Pytorch Lightning.

Thanks for your help,

[akihironitta]

I believe manual optimization is something you might want to try for your use case. See the documentation for detail on manual optimization: https://pytorch-lightning.readthedocs.io/en/1.7.2/model/manual_optimization.html?highlight=manual%20optimization#manual-optimization