Manually zero the gradients after updating weights by using machine e… #4744

Job	Run time
pytorch_tutorial_build_worker (3, 15, linux.g5.4xlarge.nvidia.gpu)	21m 18s
pytorch_tutorial_build_worker (2, 15, linux.g5.4xlarge.nvidia.gpu)	39m 17s
pytorch_tutorial_build_worker (4, 15, linux.g5.4xlarge.nvidia.gpu)	14m 0s
pytorch_tutorial_build_worker (5, 15, linux.g5.4xlarge.nvidia.gpu)	22m 23s
pytorch_tutorial_build_worker (6, 15, linux.g5.4xlarge.nvidia.gpu)	16m 41s
pytorch_tutorial_build_worker (8, 15, linux.g5.4xlarge.nvidia.gpu)	14m 40s
pytorch_tutorial_build_worker (7, 15, linux.g5.4xlarge.nvidia.gpu)	21m 3s
pytorch_tutorial_build_worker (10, 15, linux.g5.4xlarge.nvidia.gpu)	15m 37s
pytorch_tutorial_build_worker (9, 15, linux.g5.4xlarge.nvidia.gpu)	14m 7s
pytorch_tutorial_build_worker (13, 15, linux.g5.4xlarge.nvidia.gpu)	18m 59s
pytorch_tutorial_build_worker (12, 15, linux.g5.4xlarge.nvidia.gpu)	26m 31s
pytorch_tutorial_build_worker (11, 15, linux.g5.4xlarge.nvidia.gpu)	16m 24s
pytorch_tutorial_build_worker (14, 15, linux.g5.4xlarge.nvidia.gpu)	31m 52s
pytorch_tutorial_build_worker (1, 15, linux.g5.12xlarge.nvidia.gpu)	14m 43s
pytorch_tutorial_build_worker (15, 15, linux.g5.4xlarge.nvidia.gpu)	25m 38s
pytorch_tutorial_build_manager	21m 44s
	5h 34m 57s

Provide feedback