Skip to content

BUG: tmpdir directory context and ProcessPoolExecutor prevent tracing report #2065

@tylerjereddy

Description

@tylerjereddy

It seems that the combination of the pytest fixture usage tmpdir.as_cwd() and a ProcessPoolExecutor prevents the tracing of a dispatched parallel function. A very simple reproducer is possible per the code below, using Python 3.13.2, coverage 7.10.7, pytest 8.4.2. Is the coverage report likely getting lost because of the combination of an extra process and a directory change? Should this be an easy fix? Do you want me to try to patch it? Suggested workaround in the meanwhile?

To reproduce, you can copy the 3 source files below to a single directory, and run the commands:

coverage run -m pytest
coverage combine
coverage html
# probe.py
from concurrent.futures import ProcessPoolExecutor

def add(a, b):
    return a + b

def dispatcher():
    futures = []
    with ProcessPoolExecutor() as executor:
        futures.append(executor.submit(add, 2, 2))
    for future in futures:
        future.result()
# test_probe.py
from probe import dispatcher

def test_add(tmpdir):
    with tmpdir.as_cwd():
        dispatcher()

.coveragerc:

[run]
branch = true
concurrency = multiprocessing
parallel = true

And you end up with the failure to report the flushing of the dispatched function in another process:

Image

You can just deactivate the directory switching to restore a proper coverage report:

--- a/test_probe.py
+++ b/test_probe.py
@@ -1,5 +1,6 @@
 from probe import dispatcher
+import contextlib
 
 def test_add(tmpdir):
-    with tmpdir.as_cwd():
+    with contextlib.nullcontext():
         dispatcher()
Image

coverage combine only finds one data file in the problem scenario, so does indeed seem likely that one of the coverage files gets "misplaced" by the directory context. I'm happy to try to fix this if you want to provide some basic guidance--clearly it is easy to reproduce and regression test. Hopefully it is obvious why we wouldn't just remove the usage of tmpdir--we have real-world scenarios where files are being written during testing and we don't want those polluting the working directory. It is worth noting that tmpdir is no longer the preferred fixture for accomplishing this kind of thing, but in some local testing I don't see any better behavior with other approaches to switching the directory context.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions