Anyone know why the Fault-Tolerant Training document was removed from the latest pytorch-lighting document? #17811
-
|
Anyone know why the Fault-Tolerant Training document was removed from the latest pytorch-lighting document? It seems to have existed at least as of version 1.7.7. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Hey, this feature was removes with PL 2.0.0 as it was having to high impact on complexity and thereby training speed. You can still manually do this by saving the random seed of your dataloaders/datasets and restoring them upon loading. |
Beta Was this translation helpful? Give feedback.
-
|
@justusschock Thank you for your answer. I am looking for a refactor part in the Lightning League Refactored Project, but could we consider adding the manual Fault-Tolerant Training mentioned in the latter part to the document or example? I think the Lighting AI team may remove that part when it's really done. |
Beta Was this translation helpful? Give feedback.
Hey, this feature was removes with PL 2.0.0 as it was having to high impact on complexity and thereby training speed. You can still manually do this by saving the random seed of your dataloaders/datasets and restoring them upon loading.