A spurious TODO in TorchCheckpointIO

[Turakar]

Hi, I am currently working on an asynchronous version of TorchCheckpointIO that utilizes multiprocessing to save the checkpoints without blocking training. During this, I stumbled across this TODO and wondered whether it can be resolved, i.e. whether I need to duplicate that logic in my implementation?

cc @awaelchli @ananthsub @ninginthecloud @rohitgr7 @otaj

[otaj]

Hi @Turakar. The TODO might not be required anymore (we will need to test it of course before it gets removed).

However, you might be interested there's an async checkpointing in the works #13658 and @rohitgr7 is working on it to hopefully make it land in 1.7, which should happen on 1st August

[Turakar]

Thank you for the link! If you do not want to keep this issue open for removing the TODO, I am fine with closing it.

[rohitgr7]

let's keep this open and explore the TODO. @Turakar do you want to try it out?

[Turakar]

I think it seems to be related to DDP and I have no experience with that.

[awaelchli]

The TODO was resolved. Closing