Skip to content

Conversation

berland
Copy link

@berland berland commented Dec 28, 2024

If not, it is caught as a shutil.Error for which it is assumed to be a recursive error, resulting in splitting of the error string as

shutil.Error: ['<', 'D', 'i', 'r', 'E', 'n', 't', 'r', 'y', ' ', ...,
               's', 'a', 'm', 'e', ' ', 'f', 'i', 'l', 'e']

@ghost
Copy link

ghost commented Dec 28, 2024

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

bedevere-app bot commented Dec 28, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app bedevere-app bot mentioned this pull request Dec 28, 2024
@berland berland changed the title gh-102391: Catch SameFileError explicitly in _copytree gh-102931: Catch SameFileError explicitly in _copytree Dec 28, 2024
@berland
Copy link
Author

berland commented Dec 28, 2024

This is based on @eric-wieser's work in #102827 but with tests adde and a different approach to resolution.

@ZeroIntensity ZeroIntensity self-requested a review December 28, 2024 14:22
Lib/shutil.py Outdated
copy_function(srcobj, dstname)
# catch the Error from the recursive copytree so that we can
# continue with other files
except SameFileError as err:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need the same handler for SpecialFileError, which is also instantiated with a string.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(and possibly the other errors too, I haven't attempted to trace them all).

Using ExceptionGroup here might make things cleaner, but is probably backwards-incompatible.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to reproduce that with a test.

Copy link
Author

@berland berland Dec 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not make a scenario that would yield this error (a splitted string in the error message) when a SpecialFileError is triggered. There is already a test function that checks the error string when a SpecialFileError is provoked with a pipe

def test_copytree_named_pipe(self):

which already passes and that behaviour is not changed in this PR. In that case there is recursion involved, and then the existing code works. The bug with splitted strings only occurs when there is no recursion.

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some high-level comments.

@berland berland force-pushed the fix_copy_itself_exc_error branch from b8dc15b to 470b7bd Compare December 29, 2024 15:49
Lib/shutil.py Outdated
except SameFileError as err:
errors.append((srcname, dstname, str(err)))
except Error as err:
errors.extend(err.args[0])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it instead be better to use:

if type(e) is Error:
  errors.extend(err.args[0])
else:
  errors.append((srcname, dstname, str(err)))

which perhaps is more robust

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is more robust if there are more bugs that we have not found. It is best if we could find them too and add a test case for them.

If it is impossible to trigger other exceptions in the outermost recursion layer, I would prefer the code to be explicit and only catch what is attainable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer the code to be explicit and only catch what is attainable.

Agreed; perhaps then we should add

class ErrorGroup(Error): pass

and throw/catch this when grouping the exceptions. That way it is explicit that this merging behavior is actually going to do the right thinking, rather than hoping the exception is the one we think it is.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that looked good in the code! Fixed

@berland berland changed the title gh-102931: Catch SameFileError explicitly in _copytree gh-102931: Use explicit exception when _copytree should group exceptions Jan 2, 2025
@berland berland changed the title gh-102931: Use explicit exception when _copytree should group exceptions gh-102931: Use explicit exception when shutil._copytree should group exceptions Jan 2, 2025
@berland berland force-pushed the fix_copy_itself_exc_error branch 2 times, most recently from 6e9603b to 101bf86 Compare January 2, 2025 06:27
Lib/shutil.py Outdated
Comment on lines 534 to 537
except Error as err:
errors.extend(err.args[0])
errors.append((srcname, dstname, str(err)))
Copy link
Contributor

@eric-wieser eric-wieser Jan 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error extends OSError which is already caught below, so there is no need to catch this any more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. Fixed!

Copy link
Contributor

@eric-wieser eric-wieser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! It would be cool to use ExceptionGroup here, but that would probably be backwards incompatible so probably isn't possible.

This resolves a bug where a non-recursive error is assumed to be a
list of exceptions and presenting the error as:
```
shutil.Error: ['<', 'D', 'i', 'r', 'E', 'n', 't', 'r', 'y', ' ', ...,
               's', 'a', 'm', 'e', ' ', 'f', 'i', 'l', 'e']
```
@berland berland force-pushed the fix_copy_itself_exc_error branch from baff88d to 533be29 Compare January 3, 2025 06:15
@berland
Copy link
Author

berland commented Oct 14, 2025

@eric-wieser is there anything I can do to make this enter "core review"?

@berland berland requested a review from giampaolo as a code owner October 14, 2025 13:18
@python-cla-bot
Copy link

python-cla-bot bot commented Oct 14, 2025

All commit authors signed the Contributor License Agreement.

CLA signed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants