Skip to content

gh-120754: Ensure _stat_atopen is cleared on fd change #125166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Nov 1, 2024
Merged

gh-120754: Ensure _stat_atopen is cleared on fd change #125166

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
d57abd0
gh-120754: _io Ensure stat cache is cleared on fd change
cmaloney Oct 8, 2024
d2cb365
gh-120754: _pyio ensure _stat_atopen is cleared on fd change
cmaloney Oct 8, 2024
854ae91
_io: Add curlies since already modifying if
cmaloney Oct 9, 2024
c6da86e
_io: Always clear self->stat_atopen
cmaloney Oct 9, 2024
0f64483
pyio: Always clear stat_atopen just before close
cmaloney Oct 9, 2024
e2eebb0
Remove assert accidentally commited
cmaloney Oct 9, 2024
5299a04
_io: fix conditional
cmaloney Oct 9, 2024
d749709
Clear before internal_close to match _pyio os.close callsites
cmaloney Oct 22, 2024
b429587
Move to conditional free, simplify code, drop comment
cmaloney Oct 30, 2024
f08defa
Merge branch 'main' into cmaloney/fd_cleanup
cmaloney Oct 30, 2024
e153d97
Apply review fixes
cmaloney Oct 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Lib/_pyio.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1480,6 +1480,7 @@ def __init__(self, file, mode='r', closefd=True, opener=None):
"""
if self._fd >= 0:
# Have to close the existing file first.
self._stat_atopen = None
try:
if self._closefd:
os.close(self._fd)
Expand Down Expand Up @@ -1585,6 +1586,7 @@ def __init__(self, file, mode='r', closefd=True, opener=None):
except:
if owned_fd is not None:
os.close(owned_fd)
self._stat_atopen = None
raise
self._fd = fd

Expand Down Expand Up @@ -1756,6 +1758,7 @@ def close(self):
called more than once without error.
"""
if not self.closed:
self._stat_atopen = None
try:
if self._closefd:
os.close(self._fd)
Expand Down
27 changes: 17 additions & 10 deletions Modules/_io/fileio.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,8 @@
_Py_END_SUPPRESS_IPH
Py_END_ALLOW_THREADS
}
PyMem_Free(self->stat_atopen);
self->stat_atopen = NULL;
if (err < 0) {
errno = save_errno;
PyErr_SetFromErrno(PyExc_OSError);
Expand Down Expand Up @@ -258,7 +260,7 @@
#elif !defined(MS_WINDOWS)
int *atomic_flag_works = NULL;
#endif
int fstat_result;

Check warning on line 263 in Modules/_io/fileio.c

View workflow job for this annotation

GitHub Actions / Address sanitizer 

unused variable ‘fstat_result’ [-Wunused-variable]

Check warning on line 263 in Modules/_io/fileio.c

View workflow job for this annotation

GitHub Actions / Hypothesis tests on Ubuntu 

unused variable ‘fstat_result’ [-Wunused-variable]

Check warning on line 263 in Modules/_io/fileio.c

View workflow job for this annotation

GitHub Actions / Ubuntu (free-threading) / build and test (ubuntu-22.04) 

unused variable ‘fstat_result’ [-Wunused-variable]
int async_err = 0;

#ifdef Py_DEBUG
Expand All @@ -268,8 +270,9 @@
if (self->fd >= 0) {
if (self->closefd) {
/* Have to close the existing file first. */
if (internal_close(self) < 0)
if (internal_close(self) < 0) {
return -1;
}
}
else
self->fd = -1;
Expand Down Expand Up @@ -455,11 +458,17 @@
#endif
}

PyMem_Free(self->stat_atopen);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure which is preferred here. The Free + New back to back feels weird, but version with the conditional feels like "extra work/code that should never be executed" (but that the compiler / tools that run currently won't validate...).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe if (ptr != NULL) free(ptr) would be more regular and avoid the 4 lines long comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to that, dropped comment and simplified / back to always calling PyMem_New

self->stat_atopen = PyMem_New(struct _Py_stat_struct, 1);
if (self->stat_atopen == NULL) {
PyErr_NoMemory();
goto error;
/* FileIO.__init__ may be called on an already initialized object. Closing
out the old fd (see: internal_close) should always nullify
self->stat_atopen before this point. Just in case though, to prevent
leaks, only allocate a new one if required. */
assert(self->stat_atopen == NULL)
if (self->stat_atopen != NULL) {

Check failure on line 466 in Modules/_io/fileio.c

View workflow job for this annotation

GitHub Actions / Address sanitizer 

expected ‘;’ before ‘if’

Check failure on line 466 in Modules/_io/fileio.c

View workflow job for this annotation

GitHub Actions / Hypothesis tests on Ubuntu 

expected ‘;’ before ‘if’

Check failure on line 466 in Modules/_io/fileio.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / build (arm64) 

syntax error: missing ';' before 'if' [D:\a\cpython\cpython\PCbuild\_freeze_module.vcxproj]

Check failure on line 466 in Modules/_io/fileio.c

View workflow job for this annotation

GitHub Actions / Ubuntu (free-threading) / build and test (ubuntu-22.04) 

expected ‘;’ before ‘if’
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyMem_Free(NULL) is well defined: it does nothing, so you might omit the if (stat_atopen != NULL) test.

self->stat_atopen = PyMem_New(struct _Py_stat_struct, 1);
if (self->stat_atopen == NULL) {
PyErr_NoMemory();
goto error;
}
}
Py_BEGIN_ALLOW_THREADS
fstat_result = _Py_fstat_noraise(self->fd, self->stat_atopen);
Expand Down Expand Up @@ -523,10 +532,8 @@
internal_close(self);
_PyErr_ChainExceptions1(exc);
}
if (self->stat_atopen != NULL) {
PyMem_Free(self->stat_atopen);
self->stat_atopen = NULL;
}
PyMem_Free(self->stat_atopen);
self->stat_atopen = NULL;

done:
#ifdef MS_WINDOWS
Expand Down
Loading