fsdp1 -> fsdp2 (#3464) #4856

Job	Run time
pytorch_tutorial_build_worker (2, 15, linux.g5.4xlarge.nvidia.gpu)	37m 8s
pytorch_tutorial_build_worker (1, 15, linux.g5.12xlarge.nvidia.gpu)	14m 41s
pytorch_tutorial_build_worker (3, 15, linux.g5.4xlarge.nvidia.gpu)	21m 31s
pytorch_tutorial_build_worker (6, 15, linux.g5.4xlarge.nvidia.gpu)	35m 47s
pytorch_tutorial_build_worker (7, 15, linux.g5.4xlarge.nvidia.gpu)	21m 53s
pytorch_tutorial_build_worker (9, 15, linux.g5.4xlarge.nvidia.gpu)	19m 28s
pytorch_tutorial_build_worker (5, 15, linux.g5.4xlarge.nvidia.gpu)	22m 33s
pytorch_tutorial_build_worker (8, 15, linux.g5.4xlarge.nvidia.gpu)	24m 22s
pytorch_tutorial_build_worker (12, 15, linux.g5.4xlarge.nvidia.gpu)	15m 45s
pytorch_tutorial_build_worker (10, 15, linux.g5.4xlarge.nvidia.gpu)	16m 49s
pytorch_tutorial_build_worker (4, 15, linux.g5.4xlarge.nvidia.gpu)	33m 20s
pytorch_tutorial_build_worker (11, 15, linux.g5.4xlarge.nvidia.gpu)	14m 12s
pytorch_tutorial_build_worker (13, 15, linux.g5.4xlarge.nvidia.gpu)	15m 32s
pytorch_tutorial_build_worker (14, 15, linux.g5.4xlarge.nvidia.gpu)	28m 18s
pytorch_tutorial_build_worker (15, 15, linux.g5.4xlarge.nvidia.gpu)	19m 14s
pytorch_tutorial_build_manager	22m 14s
	6h 2m 47s

Provide feedback