FSDP2 tutorial: how FSDP2 works #4337

Job	Run time
pytorch_tutorial_build_worker (1, 15, linux.g5.12xlarge.nvidia.gpu)	17m 30s
pytorch_tutorial_build_worker (9, 15, linux.g5.4xlarge.nvidia.gpu)	20m 15s
pytorch_tutorial_build_worker (8, 15, linux.g5.4xlarge.nvidia.gpu)	17m 23s
pytorch_tutorial_build_worker (2, 15, linux.g5.4xlarge.nvidia.gpu)	35m 0s
pytorch_tutorial_build_worker (10, 15, linux.g5.4xlarge.nvidia.gpu)	18m 48s
pytorch_tutorial_build_worker (3, 15, linux.g5.4xlarge.nvidia.gpu)	24m 24s
pytorch_tutorial_build_worker (14, 15, linux.g5.4xlarge.nvidia.gpu)	26m 6s
pytorch_tutorial_build_worker (15, 15, linux.g5.4xlarge.nvidia.gpu)	29m 56s
pytorch_tutorial_build_worker (11, 15, linux.g5.4xlarge.nvidia.gpu)	17m 48s
pytorch_tutorial_build_worker (7, 15, linux.g5.4xlarge.nvidia.gpu)	27m 0s
pytorch_tutorial_build_worker (6, 15, linux.g5.4xlarge.nvidia.gpu)	24m 38s
pytorch_tutorial_build_worker (12, 15, linux.g5.4xlarge.nvidia.gpu)	21m 58s
pytorch_tutorial_build_worker (13, 15, linux.g5.4xlarge.nvidia.gpu)	30m 34s
pytorch_tutorial_build_worker (5, 15, linux.g5.4xlarge.nvidia.gpu)	25m 44s
pytorch_tutorial_build_worker (4, 15, linux.g5.4xlarge.nvidia.gpu)	36m 26s
pytorch_tutorial_build_manager	26m 28s
	6h 39m 58s

Provide feedback