-
-
Notifications
You must be signed in to change notification settings - Fork 33.2k
gh-127076: Ignore memory mmap in FileIO testing #127088
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
cbe85e7
gh-127086: Ignore memory mmap in FileIO testing
cmaloney 57cd5db
Change to look for MAP_ANON
cmaloney 3b46c3b
Merge branch 'main' into cmaloney/mmap_memory_file
cmaloney 8670b31
fixup comment
cmaloney e733fa0
Review fixes
cmaloney File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -72,6 +72,27 @@ def sections(self): | |
| return sections | ||
|
|
||
|
|
||
| def filter_memory(syscalls): | ||
| """Filter out memory allocation calls from File I/O calls. | ||
|
|
||
| Some calls (mmap, munmap, etc) can be used on files or to just get a block | ||
| of memory. Use this function to filter out the memory related calls from | ||
| other calls.""" | ||
|
|
||
| def _filter(call): | ||
| # mmap can operate on a fd or "MAP_ANON" which gives a block of memory. | ||
| # Ignore the "MAP_ANON" ones. | ||
| if call.syscall == "mmap" and "MAP_ANON" in call.args[3]: | ||
| return False | ||
|
||
|
|
||
| if call.syscall in ("munmap", "mprotect"): | ||
| return False | ||
|
|
||
| return True | ||
|
|
||
| return [call for call in syscalls if _filter(call)] | ||
|
|
||
|
|
||
| @support.requires_subprocess() | ||
| def strace_python(code, strace_flags, check=True): | ||
| """Run strace and return the trace. | ||
|
|
@@ -93,8 +114,6 @@ def _make_error(reason, details): | |
| "-c", | ||
| textwrap.dedent(code), | ||
| __run_using_command=[_strace_binary] + strace_flags, | ||
| # Don't want to trace our JIT's own mmap and mprotect calls: | ||
| PYTHON_JIT="0", | ||
| ) | ||
| except OSError as err: | ||
| return _make_error("Caught OSError", err) | ||
|
|
@@ -145,9 +164,14 @@ def get_events(code, strace_flags, prelude, cleanup): | |
| return all_sections['code'] | ||
|
|
||
|
|
||
| def get_syscalls(code, strace_flags, prelude="", cleanup=""): | ||
| def get_syscalls(code, strace_flags, prelude="", cleanup="", | ||
| ignore_memory=True): | ||
| """Get the syscalls which a given chunk of python code generates""" | ||
| events = get_events(code, strace_flags, prelude=prelude, cleanup=cleanup) | ||
|
|
||
| if ignore_memory: | ||
| events = filter_memory(events) | ||
|
|
||
| return [ev.syscall for ev in events] | ||
|
|
||
|
|
||
|
|
@@ -177,5 +201,5 @@ def requires_strace(): | |
| return unittest.skipUnless(_can_strace(), "Requires working strace") | ||
|
|
||
|
|
||
| __all__ = ["get_events", "get_syscalls", "requires_strace", "strace_python", | ||
| "StraceEvent", "StraceResult"] | ||
| __all__ = ["filter_memory", "get_events", "get_syscalls", "requires_strace", | ||
| "strace_python", "StraceEvent", "StraceResult"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 2 additions & 0 deletions
2
Misc/NEWS.d/next/Tests/2024-11-20-18-49-01.gh-issue-127076.DHnXxo.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| Filter out memory-related ``mmap``, ``munmap``, and ``mprotect`` calls from | ||
| file-related ones when testing :mod:`io` behavior using strace. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you move this function at the module level? Maybe rename it to _filter_memory_call().
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can move, my thought initially was "Track the address mmap returns through munmap" and the set/dictionary of "known addresses" would be in the outer scope, but I don't think
mmapiniois likely (there's themmapmodule if it's better for a particular use case)