Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions Lib/cProfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

import _lsprof
import importlib.machinery
import importlib.util
import io
import profile as _pyprofile

Expand Down Expand Up @@ -162,10 +163,8 @@ def main():
if len(args) > 0:
if options.module:
code = "run_module(modname, run_name='__main__')"
globs = {
'run_module': runpy.run_module,
'modname': args[0]
}
globs = globals().copy()
globs.update({"run_module": runpy.run_module, "modname": args[0]})
else:
progname = args[0]
sys.path.insert(0, os.path.dirname(progname))
Expand All @@ -179,10 +178,21 @@ def main():
'__name__': spec.name,
'__package__': None,
'__cached__': None,
'__builtins__': __builtins__,
}
# cmd has to run in __main__ namespace (or imports from __main__ will
# break). Clear __main__ and replace with the globals provided.
import __main__
# Save a reference to the current __main__ namespace so that we can
# restore it after cmd completes.
original_main = __main__.__dict__.copy()
__main__.__dict__.update(globs)

try:
runctx(code, globs, None, options.outfile, options.sort)
runctx(code, __main__.__dict__, None, options.outfile, options.sort)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need to restore it? You are exiting the program anyway. Also this is not ideal either. This will include all the global variables to the script that is being profiled. We want print(locals()) to be basically the same with or without the profiler.

Copy link
Contributor Author

@aneeshdurg aneeshdurg Apr 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is to juggle the global variables needed by the call to (and implementation of) runctx. It would be a lot easier if I could split cprofile into a module where __main__ only has the main function. Is that something I can do?
if not, it's still possible, just trickier/messier.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No you can't split it into its own module :( that's too much a change. I think the correct way to go is to make the full runctx path independent of any global variables.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a much cleaner fix - just ensure that the "main" function isn't run in the __main__ namespace.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm not sure if this is too hacky. I did not find such pattern in other code. It looks like an acceptable solution but I don't know if there will be implications. I want to ask @vstinner about this as he probably knows a lot of interesting usages.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this hack works as expected. But it's strange and surprising :-)

I tried but failed (test fails) to inject a new __main__ module in sys.modules and leave the cProfile module unchanged:

diff --git a/Lib/cProfile.py b/Lib/cProfile.py
index 6253755a9df..abc03fc61eb 100644
--- a/Lib/cProfile.py
+++ b/Lib/cProfile.py
@@ -166,6 +166,7 @@ def main():
                 'run_module': runpy.run_module,
                 'modname': args[0]
             }
+            modname = args[0]
         else:
             progname = args[0]
             sys.path.insert(0, os.path.dirname(progname))
@@ -181,14 +182,18 @@ def main():
                 '__cached__': None,
                 '__builtins__': __builtins__,
             }
+            modname = spec.name
+
         # cmd has to run in __main__ namespace (or imports from __main__ will
         # break). Clear __main__ and replace with the globals provided.
-        import __main__
-        __main__.__dict__.clear()
-        __main__.__dict__.update(globs)
+        import __main__ as cProfileMain
+        new_main = type(cProfileMain)(modname)
+        new_main.__dict__.clear()
+        new_main.__dict__.update(globs)
+        sys.modules['__main__'] = new_main
 
         try:
-            runctx(code, __main__.__dict__, None, options.outfile, options.sort)
+            runctx(code, new_main.__dict__, None, options.outfile, options.sort)
         except BrokenPipeError as exc:
             # Prevent "Exception ignored" during interpreter shutdown.
             sys.stdout = None

Copy link
Contributor Author

@aneeshdurg aneeshdurg Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this hack what pdb already does implicitly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dictionary of the module __main__ is used (see the explanation of

Ah, it's actually explicitly documented

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vstinner @gaogaotiantian I did a bit more poking around and I think I managed to get rid of any of the hacky-ness. cProfile's main remains untouched, and in the case where we execute a file, I create a new module, set it as main, and ensure that the globals dict is the dict of the new module. The tests pass.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the current solution looks much less hacky.

except BrokenPipeError as exc:
__main__.__dict__.clear()
__main__.__dict__.update(original_main)
# Prevent "Exception ignored" during interpreter shutdown.
sys.stdout = None
sys.exit(exc.errno)
Expand Down
20 changes: 19 additions & 1 deletion Lib/test/test_cprofile.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@

# rip off all interesting stuff from test_profile
import cProfile
import tempfile
import textwrap
from test.test_profile import ProfileTest, regenerate_expected_output
from test.support.script_helper import assert_python_failure
from test.support.script_helper import assert_python_failure, assert_python_ok
from test import support


Expand Down Expand Up @@ -155,6 +157,22 @@ def test_sort(self):
self.assertIn(b"option -s: invalid choice: 'demo'", err)


class TestProfilingScript(unittest.TestCase):
def test_profile_script_importing_main(self):
"""Check that scripts that reference __main__ see their own namespace
when being profiled."""
with tempfile.NamedTemporaryFile("w+") as f:
f.write(textwrap.dedent("""\
class Foo:
pass

import __main__
assert Foo == __main__.Foo
"""))
f.flush()
assert_python_ok('-m', "cProfile", f.name)


def main():
if '-r' not in sys.argv:
unittest.main()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Support profiling modules that import ``__main__``, such as modules that use pickle.
Loading