Model with best validation accuracy

[kanodiaayush]

Is there a way to save the model with the best validation accuracy when using early stopping? I believe right now, the model weights are the weights from the latest snapshot; but i am looking for a way to access the model with the best performance on validation.

[rohitgr7]

by on validation you mean while calling trainer.validate or validation happening within trainer.fit call?

[kanodiaayush]

I meant the latter.

[rohitgr7]

that is not possible since its part of training process. Let's say your first model is the best one, then it will just evaluate using the weights of the first checkpoint but that might not be the right way to evaluate model performance while training.

you can use ModelCheckpoint and save the best weights using that which can be later accessed.

[kanodiaayush]

I am not sure i follow the argument, because we can define best by saying we will look at validation accuracy to ascertain best, but I'm probably missing something. If we are able to use ModelCheckpoint that's good. How would one say something like 'if this is the model with best val loss' then lets checkpoint/save it? Any documentation to this kind of logic would help, not necessarily the full solution.

Thank You

[rohitgr7]

yes that is what ModelCheckpoint is for. You can configure it's arguments to make it work according to your usecase:
https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.callbacks.model_checkpoint.html

[kanodiaayush]

Thank you for this! i should have been able to find it on my own, sorry about this