How to automatically reload last checkpoint when loss unexpectedly spikes?

[RylanSchaeffer]

I'm facing a problem where during training, my loss will unexpectedly spike, like so:

When this happens, I want to automatically reload the last checkpoint, reset the optimizer and resume training. How do I do this?