Pytest code coverage does not show up

[loewexy]

Type: Bug

Behaviour

Clicking on the Run Tests with Coverage button does not have any effect. No coverage data is generated.

As you can see in the output no option enabling coverage is also passed to pytest.

I use VScode with WSL. I tested the same on a native Linux Computer and it worked as expected.

Steps to reproduce:

1. Setup any simple project with a pytest test.
2. Click on Run test with Coverage button.

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2024-10-04 10:27:29.384 [info] Default formatter is set to null for workspace /home/user/dev/test
2024-10-04 10:27:30.088 [info] Python interpreter path: ./.venv/bin/python
2024-10-04 10:27:30.533 [info] > ~/.pyenv/bin/pyenv which python
2024-10-04 10:27:30.533 [info] cwd: .
2024-10-04 10:27:32.070 [info] Discover tests for workspace name: test - uri: /home/user/dev/test
2024-10-04 10:27:32.105 [info] > pixi --version
2024-10-04 10:27:32.121 [info] > . ./.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode-server/extensions/ms-python.python-2024.16.0/python_files/printEnvVariables.py
2024-10-04 10:27:32.121 [info] shell: bash
2024-10-04 10:27:32.159 [info] > ./.venv/bin/python ~/.vscode-server/extensions/ms-python.python-2024.16.0/python_files/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear tests
2024-10-04 10:27:32.159 [info] cwd: .
2024-10-04 10:27:33.744 [info] > ./.venv/bin/python -m pytest --rootdir . --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-7959CmUhSOd99eL3.xml
2024-10-04 10:27:33.744 [info] cwd: .
2024-10-04 10:27:41.632 [info] Starting Pylance language server.

Extension version: 2024.16.0
VS Code version: Code 1.94.0 (d78a74bcdfad14d5d3b1b782f87255d802b57511, 2024-10-02T13:08:12.626Z)
OS version: Windows_NT x64 10.0.19045
Modes:
Remote OS version: Linux x64 5.15.153.1-microsoft-standard-WSL2

• Python version (& distribution if applicable, e.g. Anaconda): 3.12.2
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Poetry
• Value of the python.languageServer setting: Default

User Settings

languageServer: "Pylance"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true

Installed Extensions

Extension Name	Extension Id	Version
autoDocstring - Python Docstring Generator	njpwerner.autodocstring	0.6.1
autopep8	ms-python.autopep8	2024.0.0
Better Jinja	samuelcolvin.jinjahtml	0.20.0
Even Better TOML	tamasfe.even-better-toml	0.19.2
Flake8	ms-python.flake8	2023.10.0
Git Blame	waderyan.gitblame	11.1.0
Git Graph	mhutchie.git-graph	1.30.0
GitLab Workflow	GitLab.gitlab-workflow	5.14.0
isort	ms-python.isort	2023.10.1
JavaScript Debugger	ms-vscode.js-debug	1.94.0
Mypy Type Checker	ms-python.mypy-type-checker	2024.0.0
Pylance	ms-python.vscode-pylance	2024.9.2
Python	ms-python.python	2024.16.0
Python Debugger	ms-python.debugpy	2024.10.0
REST Client	humao.rest-client	0.25.1
Table Visualizer for JavaScript Profiles	ms-vscode.vscode-js-profile-table	1.0.10
YAML	redhat.vscode-yaml	1.15.0

System Info

Item	Value
CPUs	11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 1498)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	undefined
Memory (System)	15.73GB (4.64GB free)
Process Argv	--crash-reporter-id 4f92d2c0-8402-4937-a1b9-ca00fe63248a
Screen Reader	no
VM	40%

Item	Value
Remote	WSL: ubuntu-22-04
OS	Linux x64 5.15.153.1-microsoft-standard-WSL2
CPUs	11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 0)
Memory (System)	7.63GB (6.02GB free)
VM	0%

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492:30256859
vscod805:30301674
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonnoceb:30805159
asynctok:30898717
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
jg8ic977:31013176
a69g1124:31058053
dvdeprecation:31068756
dwnewjupyter:31046869
2f103344:31071589
impr_priority:31102340
nativerepl2:31139839
refactort:31108082
pythonrstrctxt:31112756
flightc:31134773
wkspc-onlycs-t:31132770
nativeloc2:31134642
wkspc-ranged-t:31151552
cf971741:31144450
autoexpandse:31146404
iacca2:31150323
notype1cf:31151524
cc771715:31146322

[loewexy]

It seems like the extensions does not see that pytest-cov is installed in that case. But I do not know how I can check what the reason for this behaviour is.

Running pytest with the --cov=. option directly on the cli works without problems.

[fbuessen]

I can confirm this behavior (tests run, but no coverage is generated). I'm also seeing the same behavior when running VSCode+Python natively on Windows.
@eleanorjboyd

[falkben]

Also seeing this behavior on Mac. Using conda for virtual environment

[eleanorjboyd]

Hi! From your logs it looks like you aren't on the testing rewrite which is required. Could you update to our rewrite and try again? You can do so adding this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"]. Then you can confirm its enabled by finding Experiment 'pythonTestAdapter' is active in your python logs. If you are unable to see the log for the experiment change your log level to trace and it should show up.
To set to trace use theDeveloper: set log level command in the command palette. Thanks

[fbuessen]

Hi! From your logs it looks like you aren't on the testing rewrite which is required. Could you update to our rewrite and try again? You can do so adding this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"].

Ah, the pythonTestAdapter experiment fixes the issue. Thanks!

[eleanorjboyd]

For clarity I have created a PR so the button is only surfaced on the rewrite where coverage is supported: #24249.

[loewexy]

I will check tomorrow if this also fixes the issue for me.

Did I miss this information about the required experiment in the Docs? The release notes and also the Documentation do not mention that the setting is required. And why did it work on my Linux machine out of the box?

Hiding the button is one way to handle it, if this is clearly documented I do not see an issue. The other option would be to present the user with a hint how the feature can be enabled. For me that would have been the best options, otherwise I would have opened an issue that the button is missing.

[eleanorjboyd]

yes we will add a doc update to clarify

cc @cwebster-99