Save model with best validation Error

[JonathanSchmidt1]

Despite a small learning rate I still have a quite large variation in the validation errors during training.
I would like to save the model with the smallest validation error and use that for testing and production.
Does any option for this already exist or is there interest to implement it?
It basically just requires keeping track of the validation error and saving a model under best_checkpoint.. if the validation error decreased.
Thank you very much

[wanghan-iapcm]

The on-the-fly validation randomly picks a or several batch(es) of data, and calculates the validation errors. Therefore, a small validation error printed during training does not necessary implies the model's quality is higher, unless all the validation data is used on-the-fly.