Test Discovery Failing in all repos (for the Testing Panel)

[sarahandromeda]

Type: Bug

Behaviour

When trying to use the VSCode Testing from the sidepanel, Tests are not Discoverable due to a permissions error.

I first noticed this in a single repo I was working in. I opened up another repo to see if the error persisted and it did. Then I created a separate directory with a very simple example to test and the error still occurred there as well.

Example file structure:

├── app
│   ├── __init__.py
│   └── sum_nums.py
└── tests
    ├── __init__.py
    └── test_sum_nums.py

As far as I know, nothing has changed in my system, no major updates or upgrades have happened before or since this started happening. Perhaps a chmod is needed but I am hesitant to do that without knowing why.

Why is this only occuring now? Has there been a change to the files used for the Testing execution in the last few days/weeks?

I was able to restart my computer this morning, open up a repo and run 1 test via the VSCode Testing but it only allowed me to run the singlular test before showing the error again.
I aslo recreated my Virtual Enviroment but that did not work either.

Steps to reproduce:

• Open any repo with Python code and associated tests
• Select the Testing tab in the side panel
• In side-bar, message appears Unittest Discovery Error
• Open Up the Outputs tab to Python
• See Discovery Error due to file lacking permissions (Please See Output Details Below For Outputted Error)

Tests CAN be run successfully from the command line: python -m unittest discover

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2024-05-17 09:48:08.086 [error] Unittest test discovery error for workspace: file:///home/sgomez/practice/error_example_testing 
 [Error: listen EACCES: permission denied /run/user/1000/python-test-discovery-a121c3ca49a094028295.sock
	at Server.setupListenHandle [as _listen2] (node:net:1800:21)
	at listenInCluster (node:net:1865:12)
	at Server.listen (node:net:1964:5)
	at /home/sgomez/.vscode-server/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:184541
	at new Promise (<anonymous>)
	at t.createNamedPipeServer (/home/sgomez/.vscode-server/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:184080)
	at t.startDiscoveryNamedPipe (/home/sgomez/.vscode-server/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:804427)
	at t.UnittestTestDiscoveryAdapter.discoverTests (/home/sgomez/.vscode-server/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:848795)
	at t.WorkspaceTestAdapter.discoverTests (/home/sgomez/.vscode-server/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:863627)
	at k.refreshTestDataInternal (/home/sgomez/.vscode-server/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:813783)
	at /home/sgomez/.vscode-server/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:815482
	at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
	at async Promise.all (index 0)
	at async k.resolveChildren (/home/sgomez/.vscode-server/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:815378)] {
  code: 'EACCES',
  errno: -13,
  syscall: 'listen',
  address: '/run/user/1000/python-test-discovery-a121c3ca49a094028295.sock',
  port: -1
}

Extension version: 2024.6.0
VS Code version: Code 1.89.1 (dc96b837cf6bb4af9cd736aa3af08cf8279f7685, 2024-05-07T05:13:33.891Z)
OS version: Windows_NT x64 10.0.22631
Modes:
Remote OS version: Linux x64 5.15.146.1-microsoft-standard-WSL2
Remote OS version: Linux x64 5.15.146.1-microsoft-standard-WSL2
Remote OS version: Linux x64 5.15.146.1-microsoft-standard-WSL2
Remote OS version: Linux x64 5.15.146.1-microsoft-standard-WSL2

• Python version (& distribution if applicable, e.g. Anaconda): 3.10.12
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
• Value of the python.languageServer setting: Default

User Settings

languageServer: "Pylance"

testing
• unittestEnabled: true

Installed Extensions

Extension Name	Extension Id	Version
Black Formatter	ms-python.black-formatter	2024.2.0
Cucumber (Gherkin) Full Support	alexkrechik.cucumberautocomplete	2.15.2
Docker	ms-azuretools.vscode-docker	1.29.1
EditorConfig for VS Code	EditorConfig.EditorConfig	0.16.4
Flake8	ms-python.flake8	2023.10.0
Git Prefix	srmeyers.git-prefix	1.3.2
GitLens — Git supercharged	eamodio.gitlens	15.0.3
isort	ms-python.isort	2023.10.1
JavaScript Debugger	ms-vscode.js-debug	1.89.0
json2ts	GregorBiswanger.json2ts	0.0.6
Jupyter	ms-toolsai.jupyter	2024.4.0
Jupyter Cell Tags	ms-toolsai.vscode-jupyter-cell-tags	0.1.9
Jupyter Notebook Renderers	ms-toolsai.jupyter-renderers	1.0.17
Jupyter Slide Show	ms-toolsai.vscode-jupyter-slideshow	0.1.6
Live Preview	ms-vscode.live-server	0.4.13
Move TS - Move TypeScript files and update relative imports	stringham.move-ts	1.12.0
Path Intellisense	christian-kohler.path-intellisense	2.8.5
Prettier - Code formatter	esbenp.prettier-vscode	10.4.0
Pylance	ms-python.vscode-pylance	2024.5.1
Python	ms-python.python	2024.6.0
Python Debugger	ms-python.debugpy	2024.6.0
Rainbow CSV	mechatroner.rainbow-csv	3.11.0
Table Visualizer for JavaScript Profiles	ms-vscode.vscode-js-profile-table	1.0.9
TypeScript Extension Pack	loiane.ts-extension-pack	0.3.0
TypeScript Importer	pmneo.tsimporter	2.0.1
XML Tools	DotJoshJohnson.xml	2.5.1
YAML	redhat.vscode-yaml	1.14.0

System Info

Item	Value
CPUs	AMD Ryzen 7 5825U with Radeon Graphics (16 x 1996)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	undefined
Memory (System)	15.35GB (1.02GB free)
Process Argv	--folder-uri=vscode-remote://wsl+Ubuntu/home/sgomez/kitu-batch-scripts --remote=wsl+Ubuntu --crash-reporter-id ac2a5da8-6b46-4388-bddf-1a5fe64c6b9d
Screen Reader	yes
VM	0%

Item	Value
Remote	WSL: Ubuntu
OS	Linux x64 5.15.146.1-microsoft-standard-WSL2
CPUs	AMD Ryzen 7 5825U with Radeon Graphics (16 x 1996)
Memory (System)	7.44GB (3.64GB free)
VM	0%

Item	Value
Remote	WSL: Ubuntu
OS	Linux x64 5.15.146.1-microsoft-standard-WSL2
CPUs	AMD Ryzen 7 5825U with Radeon Graphics (16 x 1996)
Memory (System)	7.44GB (3.64GB free)
VM	0%

Item	Value
Remote	WSL: Ubuntu
OS	Linux x64 5.15.146.1-microsoft-standard-WSL2
CPUs	AMD Ryzen 7 5825U with Radeon Graphics (16 x 1996)
Memory (System)	7.44GB (3.64GB free)
VM	0%

Item	Value
Remote	WSL: Ubuntu
OS	Linux x64 5.15.146.1-microsoft-standard-WSL2
CPUs	AMD Ryzen 7 5825U with Radeon Graphics (16 x 1996)
Memory (System)	7.44GB (3.64GB free)
VM	0%

A/B Experiments

vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
tftest:31042121
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
pythongtdpath:30769146
welcomedialogc:30910334
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pythonmypyd1:30879173
pythoncet0:30885854
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
showvideoc:31016891
chatpanelt:31048053
dsvsc021:30996838
0ee40948:31013168
pythoncenvpt:31049070
a69g1124:31046351
pythonprc:31047982
dwnewjupytercf:31046870
26j00206:31048877

[philipphofmann]

I have the same problem with discovering pytests. When I switch to v2024.4.1 the test discovery succeeds.

Extension version: 2024.6.0
VS Code version: Code 1.89.1 (dc96b837cf6bb4af9cd736aa3af08cf8279f7685, 2024-05-07T05:14:32.757Z)
OS version: Darwin arm64 23.4.0

Python Log Output

2024-05-21 08:36:51.437 [info] Experiment 'pythonCreateEnvOnPipInstall' is active
2024-05-21 08:36:51.437 [info] Experiment 'pythonRecommendTensorboardExt' is active
2024-05-21 08:36:51.437 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2024-05-21 08:36:51.437 [info] Experiment 'pythonTestAdapter' is active
2024-05-21 08:36:51.437 [info] Default formatter is set to ms-python.black-formatter for workspace /Users/philipp.hofmann/git-repos/sentry
2024-05-21 08:36:51.437 [info] VS Code was launched from an activated environment: '.venv', selecting it as the interpreter for workspace.
2024-05-21 08:36:51.437 [info] Python interpreter path: ./.venv/bin/python
2024-05-21 08:36:51.447 [info] > pyenv which python
2024-05-21 08:36:51.447 [info] cwd: .
2024-05-21 08:36:52.188 [info] > . ./.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.6.0/python_files/printEnvVariables.py
2024-05-21 08:36:52.188 [info] shell: zsh
2024-05-21 08:36:52.231 [info] > /usr/bin/python3 ~/.vscode/extensions/ms-python.python-2024.6.0/python_files/printEnvVariables.py
2024-05-21 08:36:52.231 [info] shell: zsh
2024-05-21 08:36:52.353 [info] Setting environment variable MANPATH in collection to /opt/homebrew/share/man:: {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-05-21 08:36:52.353 [info] Prepending environment variable PATH in collection with /Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/python_files/deactivate/zsh:/Users/philipp.hofmann/git-repos/sentry/.venv/bin: {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-05-21 08:36:52.353 [info] Prepending environment variable PS1 in collection with (.venv)  {"applyAtShellIntegration":true,"applyAtProcessCreation":false}
2024-05-21 08:36:52.353 [info] Setting environment variable VIRTUAL_ENV_PROMPT in collection to (.venv)  {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-05-21 08:36:52.354 [info] Send text to terminal: /usr/bin/python3 /Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/python_files/printEnvVariablesToFile.py /Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/python_files/deactivate/zsh/envVars.txt
2024-05-21 08:36:53.243 [info] Starting Pylance language server.
2024-05-21 08:37:06.985 [info] Discover tests for workspace name: sentry - uri: /Users/philipp.hofmann/git-repos/sentry
2024-05-21 08:37:06.986 [info] Running discovery for pytest using the new test adapter.
2024-05-21 08:37:06.992 [error] pytest test discovery error for workspace: file:///Users/philipp.hofmann/git-repos/sentry 
 Error: listen EACCES: permission denied /var/run/python-test-discovery-88f1d87a4214423346a8.sock
    at Server.setupListenHandle [as _listen2] (node:net:1800:21)
    at listenInCluster (node:net:1865:12)
    at Server.listen (node:net:1964:5)
    at /Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:184541
    at new Promise (<anonymous>)
    at t.createNamedPipeServer (/Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:184080)
    at t.startDiscoveryNamedPipe (/Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:804427)
    at t.PytestTestDiscoveryAdapter.discoverTests (/Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:829798)
    at t.WorkspaceTestAdapter.discoverTests (/Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:863627)
    at k.refreshTestDataInternal (/Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:813244)
    at /Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:815482
    at async Promise.all (index 0)
    at k.resolveChildren (/Users/philipp.hofmann/.vscode/extensions/ms-python.python-2024.6.0/out/client/extension.js:2:815378) {
  code: 'EACCES',
  errno: -13,
  syscall: 'listen',
  address: '/var/run/python-test-discovery-88f1d87a4214423346a8.sock',
  port: -1
}

System Info

Item	Value
CPUs	Apple M2 Max (12 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	3, 3, 4
Memory (System)	64.00GB (6.87GB free)
Process Argv	--crash-reporter-id ad7a646d-36a3-4c18-abc2-d460731f4f58
Screen Reader	no
VM	0%

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
tftest:31042121
vswsl492cf:30256860
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
vscaac:30438847
c4g48928:30535728
azure-dev_surveyonecf:30548226
vscrpc:30673769
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pythonmypyd1:30879173
pythoncet0:30885854
2e7ec940:31000449
pythontbext0:30879054
accentitlementsc:30995553
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
chatpanelt:31048053
dsvsc021:30996838
724cj586:31013169
pythoncenvpt:31049070
fchga941:31046352
pythonprc:31047982
dwnewjupytercf:31046870
26j00206:31048877

[senarvi]

This started happening to me after VS Code updated today.

[eleanorjboyd]

Hi! We have not made a change to unittest recently- is everyone seeing this on unittest or with pytest too? Could someone try just running unittest/pytest from the command line and seeing if they see similar permissions issues?

[sarahandromeda]

Could someone try just running unittest/pytest from the command line and seeing if they see similar permissions issues?

I only use unittest so I can't speak for pytest. However, I can run unittest from the command line with no issues. I can even run the debugger on unittest files. It is just the test discovery that is failing and leading to not being able to use the testing side panel to run anything.

[eleanorjboyd]

Interesting, it seems that your error /run/user/1000/python-test-discovery-a121c3ca49a094028295.sock is referencing our use of named pipes for communication. This is the same type of communication that run and debug use- could you check for a a similar name for the pipe in your logs for run? Secondly do you have any settings which might restrict thing? See here as we have just changed to named pipes but we do know a few issues still exist: #23279. Thanks

[senarvi]

Today, after I restarted my computer, the Testing tab opened the test configuration dialog. I selected pytest and the configuration file, and the tests appeared. This is what I see in the Python Output tab:

2024-05-22 07:20:40.215 [info] Discover tests for workspace name: ... - uri: /home/...
2024-05-22 07:20:40.215 [info] Running discovery for pytest using the new test adapter.
...
2024-05-22 07:20:40.291 [info] > . ~/.../bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode-server/extensions/ms-python.python-2024.6.0/python_files/printEnvVariables.py
2024-05-22 07:20:40.291 [info] shell: bash
2024-05-22 07:20:40.338 [info] > ~/.../bin/python -m pytest -p vscode_pytest --collect-only tests
2024-05-22 07:20:40.338 [info] cwd: .
2024-05-22 07:20:40.899 [info] ============================= test session starts ==============================
...
2024-05-22 07:20:46.290 [info] Discover tests for workspace name: ... - uri: /home/...
2024-05-22 07:20:46.290 [info] Running discovery for pytest using the new test adapter.
2024-05-22 07:20:46.290 [error] Test discovery already in progress, not starting a new one.
2024-05-22 07:21:01.510 [info] > . ~/.../bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode-server/extensions/ms-python.python-2024.6.0/python_files/printEnvVariables.py
2024-05-22 07:21:01.510 [info] shell: bash
2024-05-22 07:21:01.548 [info] > ~/.../bin/python -c "import pytest"
2024-05-22 07:21:02.206 [info] collected 184 items
...
======================== 184 tests collected in 21.33s =========================

2024-05-22 07:21:02.255 [info] Discover tests for workspace name: ... - uri: /home/.../.vscode/settings.json
2024-05-22 07:21:02.255 [info] Running discovery for pytest using the new test adapter.
2024-05-22 07:21:02.255 [error] Test discovery already in progress, not starting a new one.
2024-05-22 07:21:02.794 [info] Discover tests for workspace name: ... - uri: /home/...
2024-05-22 07:21:02.794 [info] Running discovery for pytest using the new test adapter.
2024-05-22 07:21:02.794 [error] Test discovery already in progress, not starting a new one.
2024-05-22 07:21:48.542 [info] Discover tests for workspace name: ... - uri: /home/.../tests/....py
2024-05-22 07:21:48.542 [info] Running discovery for pytest using the new test adapter.
2024-05-22 07:21:48.542 [error] pytest test discovery error for workspace: file:///home/... 
 [Error: listen EACCES: permission denied /run/user/1000/python-test-discovery-7840ce37fe6d1aa628f5.sock

There are some errors about test discovery being already in progress, and finally the permission denied error. Anyway, I can attempt running a test, but the test won't run. In Python Output tab I can see the same permission denied error for test execution:

2024-05-22 07:24:07.365 [error] pytest test execution error
 [Error: listen EACCES: permission denied /run/user/1000/python-test-results-7f1a0877cc739fa109ec.sock

My code is in WSL and I'm using pytest. I can run the tests from WSL command line using python -m pytest tests.

[philipphofmann]

I can collect the tests without problems from the command line python -m pytest --collect-only tests. It's worth noting that I use a custom venv as you can see from the logs.

/Users/philipp.hofmann/git-repos/sentry/.venv/lib/python3.11/site-packages/phabricator/__init__.py:12: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __version__ = __import__('pkg_resources') \
/Users/philipp.hofmann/git-repos/sentry/.venv/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/Users/philipp.hofmann/git-repos/sentry/.venv/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/Users/philipp.hofmann/git-repos/sentry/.venv/lib/python3.11/site-packages/pkg_resources/__init__.py:2350: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(parent)
================================================== test session starts ===================================================
platform darwin -- Python 3.11.8, pytest-8.0.0, pluggy-1.4.0
django: version: 5.0.6
rootdir: /Users/philipp.hofmann/git-repos/sentry
configfile: pyproject.toml
plugins: fail-slow-0.3.0, rerunfailures-11.0, json-report-1.5.0, metadata-3.1.1, time-machine-2.13.0, xdist-3.0.2, pytest_sentry-0.3.0, anyio-3.7.1, django-4.8.0, cov-4.0.0
collecting ...
###################
#
# Frontend assets last built 381 seconds ago, skipping rebuilds for another 3219 seconds.
# Delete the file: `.webpack.meta` to rebuild.
#
###################

[eleanorjboyd]

@karthiknadig any thoughts here on what could be causing people to suddenly have permissions issues?

[karthiknadig]

@eleanorjboyd This is the issue with using Unix Domian Socket for communication:

permission denied /run/user/1000/python-test-discovery-a121c3ca49a094028295.sock

We are working on a fix for this to use fifo pipes, than UDS pipes. For work around, see if you can allow Unix domain sockets temporarily until we get the fix out. Or, another option is to pin the extension to the older one where this works.

[sarahandromeda]

Interesting, it seems that your error /run/user/1000/python-test-discovery-a121c3ca49a094028295.sock is referencing our use of named pipes for communication. This is the same type of communication that run and debug use- could you check for a a similar name for the pipe in your logs for run? Secondly do you have any settings which might restrict thing? See here as we have just changed to named pipes but we do know a few issues still exist: #23279. Thanks

Sorry for the delay on this.

your logs for run

Do you mean in the output tab when running the tests with the debugger? (SEE BELOW)

When running the debugger on a unittest file, this is what gets logged to Output: Python

2024-05-22 15:17:54.658 [info] Send text to terminal:  /usr/bin/env /home/sgomez/my_project/.venv/bin/python /home/sgomez/.vscode-server/extensions/ms-python.debugpy-2024.6.0/bundled/libs/debugpy/adapter/../../debugpy/launcher 56803 -- /home/sgomez/my_project/tests/test_logging_helper.py 

And this it what appears in Output: Python Debugger:

2024-05-22 12:36:22.898 [info] Name: Python Debugger
2024-05-22 12:36:22.898 [info] Module: debugpy
2024-05-22 15:17:52.994 [info] DAP Server launched with command: /home/sgomez/my_project/.venv/bin/python /home/sgomez/.vscode-server/extensions/ms-python.debugpy-2024.6.0/bundled/libs/debugpy/adapter

Neither seem to reference the same pipe.

Secondly do you have any settings which might restrict thing

I don't believe so, I have not updated any settings recently. However I am using WSL on a Windows machine, but again, I have not changed any settings in my WSL or on Windows (other than system updates).