Early stopping function PyTorch

[phongsakornpoom]

I am quite new to PyTorch and I am enjoying the PyTorch for Deep Learning: Zero to Mastery book course so far. I am just wondering in "PyTorch Custom dataset" section where early stopping section. Is there any specific function, the instructor recommend to use for beginners. I searched the internet saw lot of interesting class and function surrounding the early stopping. Would be grateful if there is a specific format the Mr Daniel could recommend for beginners so I can build up from it. Thanks in advance.

[mrdbourke]

Hi @phongsakornpoom ,

Good question!

I don't have a specific function, but because PyTorch is so modular, you could create it yourself.

For example, if you wanted to perform early stopping, you could monitor the validation loss and stop training if it doesn't decrease after 5 epochs.

The following code is example of monitoring the validation loss and stopping training if it doesn't improve.

val_loss_lowest = 0

for epoch in range(epochs):
    # Do training

    # Validate
    val_loss = ...
    if epoch == 0:
        # Set the val_loss_lowest to be the val_loss for the first epoch
        val_loss_lowest = val_loss
    else:
        # If the new val_loss isn't lower than the lowest val_loss, stop training
        if not val_loss < val_loss_lowest:
            # Stop training
        else:
            val_loss_lowest = val_loss

You could try implementing it yourself and see how you go!