FSDP2 tutorial (#3358) #4432

Job	Run time
pytorch_tutorial_build_worker (5, 15, linux.g5.4xlarge.nvidia.gpu)	29m 47s
pytorch_tutorial_build_worker (4, 15, linux.g5.4xlarge.nvidia.gpu)	58m 15s
pytorch_tutorial_build_worker (1, 15, linux.g5.12xlarge.nvidia.gpu)	15m 48s
pytorch_tutorial_build_worker (2, 15, linux.g5.4xlarge.nvidia.gpu)	39m 43s
pytorch_tutorial_build_worker (3, 15, linux.g5.4xlarge.nvidia.gpu)	2h 20m 8s
pytorch_tutorial_build_worker (8, 15, linux.g5.4xlarge.nvidia.gpu)	22m 43s
pytorch_tutorial_build_worker (11, 15, linux.g5.4xlarge.nvidia.gpu)	15m 34s
pytorch_tutorial_build_worker (9, 15, linux.g5.4xlarge.nvidia.gpu)	17m 26s
pytorch_tutorial_build_worker (7, 15, linux.g5.4xlarge.nvidia.gpu)	38m 15s
pytorch_tutorial_build_worker (10, 15, linux.g5.4xlarge.nvidia.gpu)	22m 40s
pytorch_tutorial_build_worker (6, 15, linux.g5.4xlarge.nvidia.gpu)	24m 20s
pytorch_tutorial_build_worker (13, 15, linux.g5.4xlarge.nvidia.gpu)	28m 48s
pytorch_tutorial_build_worker (14, 15, linux.g5.4xlarge.nvidia.gpu)	23m 58s
pytorch_tutorial_build_worker (12, 15, linux.g5.4xlarge.nvidia.gpu)	20m 17s
pytorch_tutorial_build_worker (15, 15, linux.g5.4xlarge.nvidia.gpu)	1h 0m 44s
pytorch_tutorial_build_manager	22m 59s
	9h 41m 25s

Provide feedback