Test discovery hangs when the number of tests reaches a threshold

[jpeters71]

Type: Bug

Behaviour

The test discovery process hangs when the number of tests reaches a threshold. The threshold seems to vary by project and environment (computer/user, etc.) but there is a definite threshold. If you reduce the number of tests, eventually the test discovery will work. But if you increase the number of tests again, it will fail at the same point (for a given project/environment).

Steps to reproduce:

1. Create a really simple project.
2. Add a directory called tests to put your tests in.
3. Create a simple file with a bunch of tests like

def test_two(): pass
def test_three(): pass
...

4. Keep increasing the number of tests and restarting the test discovery process until it hangs.

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2025-01-06 12:54:41.348 [info] Discover tests for workspace name: vsctest - uri: /Users/JohnP/experiments/vsctest
2025-01-06 12:54:41.348 [info] Running discovery for pytest using the new test adapter.
2025-01-06 12:54:41.355 [info] All environment variables set for pytest discovery: {"ANDROID_SDK_ROOT":"/Users/JohnP/Library/Android/sdk","COMMAND_MODE":"unix2003","CPPFLAGS":"-I/opt/homebrew/opt/openssl@1.1/include","CRYPTOGRAPHY_SUPPRESS_LINK_FLAGS":"1","GRPC_PYTHON_BUILD_SYSTEM_OPENSSL":"1","GRPC_PYTHON_BUILD_SYSTEM_ZLIB":"1","HOME":"/Users/JohnP","HOMEBREW_CELLAR":"/opt/homebrew/Cellar","HOMEBREW_PREFIX":"/opt/homebrew","HOMEBREW_REPOSITORY":"/opt/homebrew","INFOPATH":"/opt/homebrew/share/info:","JAVA_HOME":"/Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home","LDFLAGS":"-L/opt/homebrew/opt/openssl@1.1/lib","LOGNAME":"JohnP","MANPATH":"/Users/JohnP/.nvm/versions/node/v20.5.0/share/man:/opt/homebrew/share/man:","MallocNanoZone":"0","NVM_BIN":"/Users/JohnP/.nvm/versions/node/v20.5.0/bin","NVM_DIR":"/Users/JohnP/.nvm","NVM_INC":"/Users/JohnP/.nvm/versions/node/v20.5.0/include/node","OLDPWD":"/","ORIGINAL_XDG_CURRENT_DESKTOP":"undefined","PS1":"\\[\\033[37m\\]\\W\\[\\033[34m\\]$(parse_git_branch)\\[\\033[36m\\]$\\[\\033[39m\\] ","PWD":"/Users/JohnP","SHELL":"/bin/bash","SHLVL":"1","SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.0SroJDHfZJ/Listeners","TMPDIR":"/var/folders/q0/r8z4z1016x57x1dxrdy_qqxc0000gp/T/","USER":"JohnP","VSCODE_CODE_CACHE_PATH":"/Users/JohnP/Library/Application Support/Code/CachedData/fabdb6a30b49f79a7aba0f2ad9df9b399473380f","VSCODE_CRASH_REPORTER_PROCESS_TYPE":"extensionHost","VSCODE_CWD":"/","VSCODE_ESM_ENTRYPOINT":"vs/workbench/api/node/extensionHostProcess","VSCODE_HANDLES_UNCAUGHT_ERRORS":"true","VSCODE_IPC_HOOK":"/Users/JohnP/Library/Application Support/Code/1.96-main.sock","VSCODE_NLS_CONFIG":"{\"userLocale\":\"en-us\",\"osLocale\":\"en-us\",\"resolvedLanguage\":\"en\",\"defaultMessagesFile\":\"/Applications/Visual Studio Code.app/Contents/Resources/app/out/nls.messages.json\",\"locale\":\"en-us\",\"availableLanguages\":{}}","VSCODE_PID":"16389","XPC_FLAGS":"0x0","XPC_SERVICE_NAME":"application.com.microsoft.VSCode.113271479.113271485","XTAB_ID":"-1","_":"/Applications/Visual Studio Code.app/Contents/MacOS/Electron","__CFBundleIdentifier":"com.microsoft.VSCode","__CF_USER_TEXT_ENCODING":"0x1F6:0x0:0x0","ELECTRON_RUN_AS_NODE":"1","PATH":"/Users/JohnP/.nvm/versions/node/v20.5.0/bin:/Users/JohnP/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Applications/Docker.app/Contents/Resources/bin/:/opt/homebrew/opt/php@5.6/sbin","PYTHONPATH":"/Users/JohnP/.vscode/extensions/ms-python.python-2024.22.1-darwin-arm64/python_files","TEST_RUN_PIPE":"/var/folders/q0/r8z4z1016x57x1dxrdy_qqxc0000gp/T/python-test-discovery-bc6dbf80a8051ce8e5fe"}
2025-01-06 12:54:41.360 [info] > ~/.pyenv/versions/3.9.13/bin/python -m pytest -p vscode_pytest --collect-only tests --rootdir=.
2025-01-06 12:54:41.360 [info] cwd: .
2025-01-06 12:54:41.482 [info] ============================= test session starts ==============================
platform darwin -- Python 3.9.13, pytest-7.4.0, pluggy-1.2.0
rootdir: /Users/JohnP/experiments/vsctest

2025-01-06 12:54:41.485 [info] collected 30 items

2025-01-06 12:54:41.485 [info] 
<Package tests>
  <Module test_bozo.py>
    <Function test_one>
    <Function test_two>
    <Function test_three>
    <Function test_four>
    <Function test_five>
    <Function test_six>
    <Function test_seven>
    <Function test_eight>
    <Function test_nine>

2025-01-06 12:54:41.485 [info]     <Function test_ten>
    <Function test_eleven>
    <Function test_twelve>
    <Function test_thirteen>
    <Function test_fourteen>
    <Function test_fifteen>
    <Function test_sixteen>
    <Function test_seventeen>
    <Function test_eighteen>
    <Function test_nineteen>
    <Function test_twenty>

2025-01-06 12:54:41.485 [info]     <Function test_twenty_one>
    <Function test_twenty_two>
    <Function test_twenty_three>
    <Function test_twenty_four>
    <Function test_twenty_five>
    <Function test_twenty_six>
    <Function test_twenty_seven>
    <Function test_twenty_eight>
    <Function test_twenty_nine>
    <Function test_thirty>

2025-01-06 12:54:47.190 [info] Starting Pylance language server.

Extension version: 2024.22.1
VS Code version: Code 1.96.2 (fabdb6a30b49f79a7aba0f2ad9df9b399473380f, 2024-12-19T10:22:47.216Z)
OS version: Darwin arm64 24.2.0
Modes:

• Python version (& distribution if applicable, e.g. Anaconda): 3.9.13
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Pyenv
• Value of the python.languageServer setting: Default

User Settings

languageServer: "Pylance"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true

Installed Extensions

Extension Name	Extension Id	Version
JavaScript Debugger	ms-vscode.js-debug	1.96.0
JavaScript Debugger Companion Extension	ms-vscode.js-debug-companion	1.1.3
Pylance	ms-python.vscode-pylance	2024.12.1
Python	ms-python.python	2024.22.1
Python Debugger	ms-python.debugpy	2024.14.0
Table Visualizer for JavaScript Profiles	ms-vscode.vscode-js-profile-table	1.0.10

System Info

Item	Value
CPUs	Apple M1 Pro (10 x 2400)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	3, 2, 2
Memory (System)	16.00GB (0.20GB free)
Process Argv	--crash-reporter-id 9ff1e562-86d4-4584-be69-ad043ea6302b
Screen Reader	no
VM	0%

A/B Experiments

vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyt551:31179978
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythonnoceb:30805159
pythonmypyd1:30879173
2e7ec940:31000449
pythontbext0:30879054
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
dvdeprecation:31068756
dwnewjupytercf:31046870
2f103344:31071589
nativerepl2:31139839
pythonrstrctxt:31112756
nativeloc2:31192216
cf971741:31144450
iacca1:31171482
notype1cf:31157160
5fd0e150:31155592
dwcopilot:31170013
stablechunks:31184530
6074i472:31201624

[eleanorjboyd]

duplicate of #24656, sorry for the problem and we are working on a fix! Thanks

[jpeters71]

Thanks for the update and workaround!