Change Optimizer when validation loss stops improving

[athenawisdoms]

Hello, how can we configure Lightning to switch from one optimizer to the second optimizer (e.g. Adam to SGD) when the validation loss stops improving for a certain number of validation epochs?

Kind of like EarlyStopping and its patience parameter, but switches optimizer instead of stop training.

Thank you!

[awaelchli]

Hi
This is not so easy, because the optimizer configuration setup depends on the training plugins being used.
It would be a useful feature though.
Could you file a feature request on the issues page here on github?

[athenawisdoms]

Thanks! Opened issue #6829