Multi-GPU Distributed Training Process Getting Stuck #11
Description
Hi, Yinyu! I am currently working with your ScenePriors repository and attempting to implement multi-GPU distributed training. However, I have encountered an issue where one of the training processes becomes stuck and does not progress along with the others, as shown in the following figure.
If you have previously experienced this issue or if you are aware of any potential reasons that might cause a process to hang during distributed training with multiple GPUs?
Thanks a lot.
Best,
Xin