[Bug]: llama-dev test suite fails in local mode due to invalid mocked package paths

[CoralGarden52]

Bug description

Running the llama-dev test suite locally fails in the local Rich progress-display path because the tests use mocked package paths outside repo_root.

Environment

• Commit: afd0fef
• Python: 3.13.12
• pytest: 8.3.5
• uv: 0.11.6
• OS: Linux
• llama-dev: 0.1

Steps to reproduce

From the repository root:

1. Change into the llama-dev package:

   cd llama-dev

2. Run the test suite without CI environment variables:

   env -u CI -u GITHUB_ACTIONS uv run pytest -q

Actual result

The test suite reports 5 failures and 64 passes. All failures raise:

ValueError: '/fake/repo/package1' is not in the subpath of
'/.../llama-dev/tests/data'

The failing tests are:

• test_coverage_failures
• test_install_failures
• test_skip_failures_no_tests
• test_skip_failures_unsupported_python
• test_success

Expected result

The local test suite should complete successfully and report the expected mocked test results.

Relevant code

• llama-dev/llama_dev/test/__init__.py:227
• llama-dev/llama_dev/test/__init__.py:252
• llama-dev/tests/test/test_test.py:146

The tests configure repo_root as the tests/data directory but return /fake/repo/package1 from mocked package discovery. The local Rich renderer calls Path.relative_to(repo_root) and raises before the mocked result can be displayed.

The issue is masked in CI because GitHub Actions sets CI=true and GITHUB_ACTIONS=true, which selects a different progress-reporting branch. For comparison:

CI=true uv run pytest tests/test/test_test.py -q

reports 20 passed.

[dosubot]

@CoralGarden52 good analysis. The root cause is that monkeypatch.setattr on _run_tests only takes effect in the parent process, but ProcessPoolExecutor spawns child processes that import the module fresh — so the monkeypatch never reaches the workers [1].

Here's the full chain:

1. The five failing tests (test_coverage_failures, test_install_failures, test_skip_failures, test_success) all mock get_changed_packages to return {Path("/fake/repo/package1")} [2]. This becomes packages_to_test via changed_packages | dependants [3].

2. The ProcessPoolExecutor submits _run_tests with Path("/fake/repo/package1") as the package_path argument [1]. Because the executor runs in a child process, the monkeypatch.setattr(llama_dev_test, "_run_tests", mocked_success) applied in the parent has no effect — the real _run_tests executes instead [4].

3. The real _run_tests returns {"package": package_path, ...} — i.e. Path("/fake/repo/package1") [5]. Back in the parent, package.relative_to(repo_root) tries to compute Path("/fake/repo/package1").relative_to(tests/data/), which raises ValueError [6].

4. This is masked in CI because environment variables like CI=true / GITHUB_ACTIONS=true likely select a different code path that bypasses the Rich progress display where relative_to is called.

Fix options (smallest to cleanest):

• Mock ProcessPoolExecutor entirely (like test_workers_parameter already does): add @mock.patch("llama_dev.test.concurrent.futures.ProcessPoolExecutor") to the failing tests and wire up future.result() to return the mocked result dicts directly. This is the cleanest approach — it avoids the cross-process monkeypatching limitation and is already the established pattern in this test file.

• Fix mock return values: change the mocked functions to return "package": Path("/fake/repo/package1") (matching what the executor actually submits) and use --repo-root /fake/repo instead of data_path. This keeps the executor real but requires the fake paths to be internally consistent.

• Make the executor injectable: allow test() to accept an executor class so tests can swap in ThreadPoolExecutor (where monkeypatching works across threads in the same process).

The first option is the lowest-friction fix since it follows the existing pattern in the same file.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.