why "File Uncovered" for my tests_*.py inside `tests/` folder?

[alanwilter]

I use pyproject.toml and I have:

[tool.pytest.ini_options]
addopts = "tests --cov-branch --cov=src/pdfgen/ --cov=tests/ --cov-report=term-missing:skip-covered --cov-report=xml --dist=loadgroup -n 8 --durations=5"

When I run pytest on my vscode terminal, everything is fine, there's not even branches missing. But when I open a test_main.py file for example, Gutters reports File Uncovered, why? What am I missing?

FYI, openning src/pdfgen/main.py shows the gutters pretty fine.

[mattseddon]

@alanwilter what do you expect to see? Would you expect the test files to be 100% covered? They are obviously executed by the test runner but I would assume they are excluded by default.

[alanwilter]

I hope this can help:

tests/test_integration.py:

No gutters, "File Uncovered". It should show "100%" or at least some value.

src/myapp/main.py:

All fine.

When I use Vscode Test Coverage, it shows everything is fine and covered, including tests

[mattseddon]

In the context of pytest-cov (which uses coverage.py), "covered" refers to the executable statements or branches within your Python code that have been executed during a test run. By definition, your test_* files are 100% covered as they contain the tests that being executed during the run. For this reason, I don't see why you'd want to see coverage displayed for these files.

This extension should, however, display the information from the provided coverage file. Can you please check the XML file to confirm whether or not it contains information about the test_ files.

[alanwilter]

One can't assume that test_*.py will be fully covered. Thanks to this new feature in Vscode, I was able to spot tests that were useless, using deprecated or removed functions, or non-used functions in the test files, and I found this very useful! So, I don't understand why gutters does not "cover" the test files (or actually, any file in tests/ folder). I don't see anything wrong with my coverage.xml.

In any case, I have this in my pyproject.toml:

[tool.pytest.ini_options]
addopts = "tests --cov-branch --cov=src/mbpdfgen/ --cov=tests/ --cov-report=term-missing:skip-covered --cov-report=xml --dist=loadgroup -n 0 --durations=5"

[tool.coverage.report]
omit = ["__main__.py", "download_binaries.py"]
exclude_lines = ["no cov", "if __name__ == .__main__.:", "if TYPE_CHECKING:"]

I run pytest and I get this:

$ pytest
========================================= test session starts ==========================================
platform darwin -- Python 3.12.11, pytest-8.4.2, pluggy-1.6.0
rootdir: /..../Projects/mbpdfgen
configfile: pyproject.toml
plugins: xdist-3.8.0, cov-7.0.0
collected 32 items                                                                                     

tests/test_binary.py ........                                                                    [ 25%]
tests/test_integration.py ..                                                                     [ 31%]
tests/test_main.py ..............                                                                [ 75%]
tests/test_typst_builder.py ........                                                             [100%]

============================================ tests coverage ============================================
_______________________________________________ coverage _______________________________________________

Name    Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------
TOTAL     427      0     46      0   100%

9 files skipped due to complete coverage.
Coverage XML written to file coverage.xml
========================================= slowest 5 durations ==========================================
0.92s call     tests/test_integration.py::test_full_pdf_generation
0.81s call     tests/test_main.py::test_main_help_flag
0.17s call     tests/test_integration.py::test_pdf_generation_with_custom_template
0.05s setup    tests/test_integration.py::test_full_pdf_generation
0.01s setup    tests/test_main.py::test_main_successful_execution
========================================== 32 passed in 6.16s ==========================================

Now, if I add, in a test file or in conftest.py, something like this:

def this_function_is_empty() -> None:
    return None

And run pytest or Vscode "Run Tests with Coverage", they will spot this useless code:

Name                Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------
tests/conftest.py      57      1      2      0    98%   122
---------------------------------------------------------------
TOTAL                 429      1     46      0    99%

But "Gutters" is none the wiser about it.

[mattseddon]

@alanwilter no worries. Happy to review a PR if you raise one to fix the issue 👍🏻.