multi-root workspace python test discovery hangs with more than 3 projects

[c0state]

Type: Bug

Behaviour

I have a multi-root workspace with a few Python folders in it. When I add up to 3 of these folders, test discovery works and the test explorer shows the test tree view. When I add another folder, discovery hangs with no errors logged. The project folder itself isn't the issue. Mixing and matching up to 3 work fine.

Steps to reproduce:

1. Create a multi-root workspace and add up to 3 Python project folders to it.
2. Confirm that test discovery works. Add another Python folder and observe that it hangs with no output indicating what the error is.
3. Remove one of the previous 3 working folders to confirm that the new project folder itself isn't the issue.

Interestingly, in the "Python output window" section (see output below), All environment variables set for pytest discovery is only emitted 3 times so it presumably hung up trying to "process" the 4th project.

There would seem to be some sort of resource issue causing this. The logs are too sparse to indicate what however.

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2024-12-12 12:31:57.548 [info] Discover tests for workspace name: carrier - uri: /Users/c0state/work/proj1
2024-12-12 12:31:57.548 [info] Running discovery for pytest using the new test adapter.
2024-12-12 12:31:57.548 [info] Discover tests for workspace name: conclave - uri: /Users/c0state/work/proj2
2024-12-12 12:31:57.548 [info] Running discovery for pytest using the new test adapter.
2024-12-12 12:31:57.548 [info] Discover tests for workspace name: consular - uri: /Users/c0state/work/proj3
2024-12-12 12:31:57.548 [info] Running discovery for pytest using the new test adapter.
2024-12-12 12:31:57.549 [info] Discover tests for workspace name: podracer - uri: /Users/c0state/work/proj4
2024-12-12 12:31:57.549 [info] Running discovery for pytest using the new test adapter.
2024-12-12 12:31:57.678 [info] All environment variables set for pytest discovery: { ... }

Extension version: 2024.22.0
VS Code version: Code 1.96.0 (138f619c86f1199955d53b4166bef66ef252935c, 2024-12-11T02:29:09.626Z)
OS version: Darwin arm64 24.2.0
Modes:

• Python version (& distribution if applicable, e.g. Anaconda): 3.12.7
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
• Value of the python.languageServer setting: Default

User Settings

Multiroot scenario, following user settings may not apply:

languageServer: "Pylance"

testing
• promptToConfigure: false
• pytestEnabled: true

terminal
• activateEnvironment: false

Installed Extensions

Extension Name	Extension Id	Version
Apollo GraphQL	apollographql.vscode-apollo	2.5.1
C/C++	ms-vscode.cpptools	1.22.11
C/C++ Extension Pack	ms-vscode.cpptools-extension-pack	1.3.0
C/C++ Themes	ms-vscode.cpptools-themes	2.0.0
CMake	twxs.cmake	0.0.17
CMake Tools	ms-vscode.cmake-tools	1.19.52
Debugger for Firefox	firefox-devtools.vscode-firefox-debug	2.11.0
Dev Containers	ms-vscode-remote.remote-containers	0.394.0
Docker	ms-azuretools.vscode-docker	1.29.3
ESLint	dbaeumer.vscode-eslint	3.0.10
Flake8	ms-python.flake8	2023.10.0
GitHub Actions	github.vscode-github-actions	0.27.0
GitHub Copilot	GitHub.copilot	1.250.0
GitHub Copilot Chat	GitHub.copilot-chat	0.23.1
GitHub Pull Requests	GitHub.vscode-pull-request-github	0.102.0
GitHub Theme	GitHub.github-vscode-theme	6.3.5
GitLens — Git supercharged	eamodio.gitlens	16.0.5
Go	golang.go	0.42.1
JavaScript Debugger	ms-vscode.js-debug	1.96.0
JavaScript Debugger Companion Extension	ms-vscode.js-debug-companion	1.1.3
Jupyter	ms-toolsai.jupyter	2024.11.0
Jupyter Cell Tags	ms-toolsai.vscode-jupyter-cell-tags	0.1.9
Jupyter Keymap	ms-toolsai.jupyter-keymap	1.1.2
Jupyter Notebook Renderers	ms-toolsai.jupyter-renderers	1.0.21
Jupyter Slide Show	ms-toolsai.vscode-jupyter-slideshow	0.1.6
Kubernetes	ms-kubernetes-tools.vscode-kubernetes-tools	1.3.18
Prettier - Code formatter	esbenp.prettier-vscode	11.0.0
Pylance	ms-python.vscode-pylance	2024.12.1
Python	ms-python.python	2024.22.0
Python Debugger	ms-python.debugpy	2024.14.0
Remote - SSH	ms-vscode-remote.remote-ssh	0.116.0
Remote - SSH: Editing Configuration Files	ms-vscode-remote.remote-ssh-edit	0.87.0
Remote Explorer	ms-vscode.remote-explorer	0.4.3
Table Visualizer for JavaScript Profiles	ms-vscode.vscode-js-profile-table	1.0.10
Vim	vscodevim.vim	1.29.0
YAML	redhat.vscode-yaml	1.15.0
ZipFS - a zip file system	arcanis.vscode-zipfs	3.0.0

System Info

Item	Value
CPUs	Apple M1 Max (10 x 2400)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	99, 123, 74
Memory (System)	64.00GB (14.25GB free)
Process Argv	--crash-reporter-id 0c627b18-87e6-4890-ad4f-ed62d421ac06
Screen Reader	no
VM	0%

A/B Experiments

vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
pythonnoceb:30805159
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
dvdeprecation:31068756
dwnewjupyter:31046869
newcmakeconfigv2:31071590
nativerepl2:31139839
pythonrstrctxt:31112756
nativeloc1:31192215
cf971741:31144450
iacca1:31171482
notype1cf:31157160
5fd0e150:31155592
dwcopilot:31170013
stablechunks:31184530

[asanglard]

same problem here. this is clearly a regression of the last version v2024.22.0.

2024.20.0 works fine.

I spent half day to find the root cause...

[troshlyak]

I can also confirm that I have exactly the same behaviour. We have Workspace with 27 folders:

• Commenting out folders down to three that have python tests enabled works
• Uncommenting folders that does not have python tests does not affect the test discovery
• Uncommenting 4th folder with python tests hangs the test discovery

VSCode version: Version: 1.96.0
OS version: Darwin arm64 24.1.0

Current workaround for me was to revert to the previous version "2024.20.0" and disable Auto Update for this extension for now.

[fensoft]

I can also confirm I have the exact same problem.

VSCode 1.96.0
arm64 23.6.0 Darwin (Sonoma 14.6.1)
ms-python.python 2024.22.0

I have cleaned everything but installed only python extension:

rm -fr ~/Library/Preferences/com.microsoft.VSCode.helper.plist
rm -fr ~/Library/Preferences/com.microsoft.VSCode.plist
rm -fr ~/Library/Caches/com.microsoft.VSCode
rm -fr ~/Library/Caches/com.microsoft.VSCode.ShipIt/
rm -fr ~/Library/Application\ Support/Code/
rm -fr ~/Library/Saved\ Application\ State/com.microsoft.VSCode.savedState/
rm -fr ~/.vscode/

[ollyhensby]

same problem here. this is clearly a regression of the last version v2024.22.0.

2024.20.0 works fine.

I spent half day to find the root cause...

Thank you! Started having this problem today also. Reverting to 2024.20.0 fixed it for me.

[carschno]

I see the same issue. Possibly related: my workspace includes projects for which pytest discovery fails due to issues in the projects. Until vscode-python version 2024.20.0, these errors are reported, and other projects are discovered correctly.

[tboddyspargo]

@eleanorjboyd - I don't know about others, but I'm still experiencing a situation where the Test Explorer gets stuck and stops enumerating discovered tests (or test discovery itself is stuck...). Some (~7) of my workspace folders populate with tests, 2 of them show errors, but the loading indicator never goes away and the rest of them (~10) never populate.

I'm on ms-python.python version 2024.23.2024121801, which I think should include the fix for this issue (?).

My work-around is to optOut of the pytestAdapter experiment and revert back to 2024.20.0.

I noticed this in the logs (although, it seems unlikely to be the root cause here):

2024-12-17 16:47:45.417 [warning] Unknown command line option passed into args parser for tests '-vv'. Please report on https://github.com/Microsoft/vscode-python/issues/new

So I removed -vv from my pytestArgs (I normally want this so that my Test Results output shows me full diffs of test failures), but then I see this error in the logs despite waiting multiple minutes:

2024-12-18 10:03:10.609 [error] Test discovery already in progress, not starting a new one.

After restarting VSCode to see how it works without -vv, the same issue persists. From the logs, it seems that all (or almost all? not sure - it's hard to correlate all the output with the correct folder) discovery processes finish, but the Test Explorer is stuck. Is there perhaps some disconnect between the background processes and the "listeners" for them? Do they become "misaligned" after one discovery hits an error.

I'll note that, in the past, test discovery and test run has had a harder time recovering from import-time Exceptions/failures (e.g. ModuleNotFoundError: No module named '...') than test/fixture Exceptions. Could that be related?

We can spin this off into a separate issue, if that would be helpful!

[carschno]

@eleanorjboyd - I don't know about others, but I'm still experiencing a situation where the Test Explorer gets stuck and stops enumerating discovered tests (or test discovery itself is stuck...).

@tboddyspargo There has not been a release since the fix. I understand it is planned for January 2025.
The latest release is still v2024.22.0.

[tboddyspargo]

@tboddyspargo There has not been a release since the fix. I understand it is planned for January 2025.
The latest release is still v2024.22.0.

I understand that PR has been made available in the pre-release versions (including 2024.23.2024121801), which I've tried. Am I mistaken?