Train model for a few iteration and continue

[we-taper]

Hi pytorch-lightning community,

I want to be able to train my model for a few iterations, pause, check for some outputs, and continue from that.
It seems that this is not natively supported in PyTorch lightning's Trainer.fit.
Currently, I achieve this by:

pl_trainer = pl.Trainer(max_epochs=0)  # max_epochs will be increase dynamically later
# say I want to run 10 iterations
pl_trainer.fit_loop.max_epochs += 10
pl_trainer.fit(...)
# then, maybe I want to run another 20 iterations
pl_trainer.fit_loop.max_epochs += 20
pl_trainer.fit(...)

However, this isn't ideal since the trainer always calls finalize on my logger, which sometimes messes up the logger.
For example, for Aim's logger, it clears the current experiment, and starts a new one for the second 20 iterations.

Are there any alternatives?