Fix race condition in FileSystemPackageRepository directory creation, modernize usage of os.makedirs#1913
Conversation
Signed-off-by: Nathan Rusch <nathan.rusch@scanlinevfx.com>
Signed-off-by: Nathan Rusch <nathan.rusch@scanlinevfx.com>
Signed-off-by: Nathan Rusch <nathan.rusch@scanlinevfx.com>
|
Thanks to @mgcollie for pointing out the original race condition. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1913 +/- ##
==========================================
+ Coverage 60.05% 60.07% +0.01%
==========================================
Files 163 163
Lines 20120 20096 -24
Branches 3507 3493 -14
==========================================
- Hits 12084 12072 -12
+ Misses 7216 7208 -8
+ Partials 820 816 -4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Thanks @nrusch. I agree with you that the semantics of safe_makedirs seem to be almost the same as os.makedirs(path, exists_ok=True), at least according to https://github.com/python/cpython/blob/b14986c91464b06e9016678637e41d0e0192bbe8/Lib/os.py#L211-L241, which hasn't changed since at least 3.7. There are small differences, but nothing major, only improvements.
0c4b28f
into
AcademySoftwareFoundation:main
The original impetus for this PR was to fix a race condition in
FileSystemPackageRepository.pre_variant_installif multiple processes attempted to create a family directory in a narrow timing window.From there, I decided to also go through and "modernize" any usage of
os.makedirsto leverage the newexist_okkwarg added in Python 3.2.The PR is split up into 3 commits, in case you'd prefer to only integrate a subset:
rez.utils.filesystem.safe_makedirswith directos.makedirscalls.os.makedirsto useexist_ok=Truewhere it seems appropriate.