FullyShardedDataParallel with 3D segmentation

[clarkbab]

Hi there,

I originally posted this question in the PyTorch forums because it is related to use of the FullyShardedDataParallel, but I thought it would be worthwhile reposting here.

I'm trying to adapt the BRATS 3D segmentation tutorial to run using multiple GPUs to reduce the memory footprint and hence train using higher-resolution input images. Has anybody had any success with this?

Please see the original post for full details.

Thanks,
Brett