Improve Fault Tolerance via TorchFT #20967
Description
Description & Motivation
https://github.com/pytorch/torchft
https://pytorch.org/blog/fault-tolerant-llama-training-with-2000-synthetic-failures-every-15-seconds-and-no-checkpoints-on-crusoe-l40s/
I would like there to be able to use torchft to improve my fault tolerance inside of pytorch lightning.
Pitch
No response
Alternatives
No response
Additional context
No response