[mypyc] Add hidden flag to skip the generation of C files (#18955)

This can be useful when debugging mypyc issues. For example, you can
manually add some debug prints to the generated C and rerun mypyc
with `--skip-c-gen`. Now mypyc will build the C code again, with your
manual changes included (this assumes everything else is the same as
during the previous run).

I'm not planning to advertise this as an end-user feature.

Author: JukkaL

mypy/main.py

@@ -1128,6 +1128,11 @@ def add_invertible_flag(
     report_group.add_argument(
         "-a", dest="mypyc_annotation_file", type=str, default=None, help=argparse.SUPPRESS
     )
+    # Hidden mypyc feature: do not write any C files (keep existing ones and assume they exist).
+    # This can be useful when debugging mypyc bugs.
+    report_group.add_argument(
+        "--skip-c-gen", dest="mypyc_skip_c_generation", action="store_true", help=argparse.SUPPRESS
+    )
 
     other_group = parser.add_argument_group(title="Miscellaneous")
     other_group.add_argument("--quickstart-file", help=argparse.SUPPRESS)

mypy/options.py

@@ -408,6 +408,9 @@ def __init__(self) -> None:
 
         # Output html file for mypyc -a
         self.mypyc_annotation_file: str | None = None
+        # Skip writing C output files, but perform all other steps of a build (allows
+        # preserving manual tweaks to generated C file)
+        self.mypyc_skip_c_generation = False
 
     def use_lowercase_names(self) -> bool:
         if self.python_version >= (3, 9):

mypyc/build.py

@@ -452,7 +452,8 @@ def mypyc_build(
         cfilenames = []
         for cfile, ctext in cfiles:
             cfile = os.path.join(compiler_options.target_dir, cfile)
-            write_file(cfile, ctext)
+            if not options.mypyc_skip_c_generation:
+                write_file(cfile, ctext)
             if os.path.splitext(cfile)[1] == ".c":
                 cfilenames.append(cfile)
 

mypyc/doc/dev-intro.md

@@ -386,6 +386,25 @@ Test cases can also have a `[out]` section, which specifies the
 expected contents of stdout the test case should produce. New test
 cases should prefer assert statements to `[out]` sections.
 
+### Adding Debug Prints and Editing Generated C
+
+Sometimes it's helpful to add some debug prints or other debugging helpers
+to the generated C code. You can run mypyc using `--skip-c-gen` to skip the C
+generation step, so all manual changes to C files are preserved. Here is
+an example of how to use the workflow:
+
+* Compile some file you want to debug: `python -m mypyc foo.py`.
+* Add debug prints to the generated C in `build/__native.c`.
+* Run the same compilation command line again, but add `--skip-c-gen`:
+  `python -m mypyc --skip-c-gen foo.py`. This will only rebuild the
+  binaries.
+* Run the compiled code, including your changes: `python -c 'import foo'`.
+  You should now see the output from the debug prints you added.
+
+This can also be helpful if you want to quickly experiment with different
+implementation techniques, without having to first figure out how to
+modify mypyc to generate the desired C code.
+
 ### Debugging Segfaults
 
 If you experience a segfault, it's recommended to use a debugger that supports