Fix flaky test_directory_path_synced UT (#878)

diego-urgell · facebook-github-bot · commit 0efa62b5d71a · 2024-08-13T10:55:55.000-07:00
Summary: Pull Request resolved: #878 Reviewed By: JKSenthil Differential Revision: D61165268 fbshipit-source-id: d2dc1987d02f18804ace7d9ac5c1a9b9a33f946d
diff --git a/tests/framework/callbacks/test_base_checkpointer.py b/tests/framework/callbacks/test_base_checkpointer.py
@@ -1084,6 +1084,7 @@ def _test_directory_path_synced() -> None:
                 ["epoch_10_train_step_0"],
             )
         finally:
+            dist.barrier()  # avoid race condition
             if get_global_rank() == 0:
                 shutil.rmtree(temp_dir)  # delete temp directory
 

Original file line number	Diff line number	Diff line change
`@@ -1084,6 +1084,7 @@ def _test_directory_path_synced() -> None:`
`1084`	`1084`	`["epoch_10_train_step_0"],`
`1085`	`1085`	`)`
`1086`	`1086`	`finally:`
	`1087`	`+ dist.barrier() # avoid race condition`
`1087`	`1088`	`if get_global_rank() == 0:`
`1088`	`1089`	`shutil.rmtree(temp_dir) # delete temp directory`
`1089`	`1090`