-
-
Notifications
You must be signed in to change notification settings - Fork 33.2k
gh-102931: Use explicit exception when shutil._copytree should group exceptions #128309
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
This is based on @eric-wieser's work in #102827 but with tests adde and a different approach to resolution. |
Lib/shutil.py
Outdated
copy_function(srcobj, dstname) | ||
# catch the Error from the recursive copytree so that we can | ||
# continue with other files | ||
except SameFileError as err: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you need the same handler for SpecialFileError
, which is also instantiated with a string.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(and possibly the other errors too, I haven't attempted to trace them all).
Using ExceptionGroup
here might make things cleaner, but is probably backwards-incompatible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will try to reproduce that with a test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could not make a scenario that would yield this error (a splitted string in the error message) when a SpecialFileError is triggered. There is already a test function that checks the error string when a SpecialFileError is provoked with a pipe
cpython/Lib/test/test_shutil.py
Line 999 in ffece55
def test_copytree_named_pipe(self): |
which already passes and that behaviour is not changed in this PR. In that case there is recursion involved, and then the existing code works. The bug with splitted strings only occurs when there is no recursion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some high-level comments.
Misc/NEWS.d/next/Library/2024-12-28-16-25-42.gh-issue-102931.55o5kb.rst
Outdated
Show resolved
Hide resolved
b8dc15b
to
470b7bd
Compare
Lib/shutil.py
Outdated
except SameFileError as err: | ||
errors.append((srcname, dstname, str(err))) | ||
except Error as err: | ||
errors.extend(err.args[0]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it instead be better to use:
if type(e) is Error:
errors.extend(err.args[0])
else:
errors.append((srcname, dstname, str(err)))
which perhaps is more robust
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it is more robust if there are more bugs that we have not found. It is best if we could find them too and add a test case for them.
If it is impossible to trigger other exceptions in the outermost recursion layer, I would prefer the code to be explicit and only catch what is attainable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would prefer the code to be explicit and only catch what is attainable.
Agreed; perhaps then we should add
class ErrorGroup(Error): pass
and throw/catch this when grouping the exceptions. That way it is explicit that this merging behavior is actually going to do the right thinking, rather than hoping the exception is the one we think it is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, that looked good in the code! Fixed
6e9603b
to
101bf86
Compare
Lib/shutil.py
Outdated
except Error as err: | ||
errors.extend(err.args[0]) | ||
errors.append((srcname, dstname, str(err))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Error
extends OSError
which is already caught below, so there is no need to catch this any more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup. Fixed!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! It would be cool to use ExceptionGroup
here, but that would probably be backwards incompatible so probably isn't possible.
This resolves a bug where a non-recursive error is assumed to be a list of exceptions and presenting the error as: ``` shutil.Error: ['<', 'D', 'i', 'r', 'E', 'n', 't', 'r', 'y', ' ', ..., 's', 'a', 'm', 'e', ' ', 'f', 'i', 'l', 'e'] ```
baff88d
to
533be29
Compare
@eric-wieser is there anything I can do to make this enter "core review"? |
If not, it is caught as a shutil.Error for which it is assumed to be a recursive error, resulting in splitting of the error string as