Test runner with unittest seems to ignore updates to tests once decorated to skip

[irenepc]

Type: Bug

Behaviour

I'm using unittest to test python code within a multi-root project. Only one of those roots is correctly discovered, but that's expected because of the configuration.

My issue is that the testing tools seem not to respect or correctly process tests and changes to tests that have been decorated to skip. Even closing and reopening vscode does not update whether a test should be skipped after re-discovering the tests.

There's two issues I'm seeing. The first is that adding raise unittest.SkipTest() in a TestCase derived class within its setUpClass method seems to be ignored when running tests.

The second is that adding this code to a class, also breaks updating skipping/not skipping on the contained tests. So if a decorator is added to skip, or removed, the test continues to run or get skipped, respectively.

Steps to reproduce:

1. In a multi-root project (if relevant)
2. Set up a test class so that it raises to skip in its set up class method, like so:

    @classmethod
    def setUpClass(cls):
        raise unittest.SkipTest()

3. Add some fake tests and run
4. Tests will not be skipped
5. decorate one of the tests with @unittest.skip("skip")
6. Test will still not be skipped
7. Close and reopen vscode and re-discover tests
8. Test will STILL not be skipped
9. Now remove the raise in setUpClass.
10. Tests will now correctly be skipped if they're decorated and vice versa.

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2025-09-25 05:24:45.509 [info] Running UNITTEST execution for the following test ids: ttfenv.perforce.tests.test_perforce_admin_utils.TestPerforceUtilitiesNotDry.test_create_client,ttfenv.perforce.tests.test_perforce_admin_utils.TestPerforceUtilitiesNotDry.test_delete_client
2025-09-25 05:24:45.510 [info] Attempting to use temp directory for test ids file, file name: test-ids-1f57f2a1f5e77c3d7348.txt
2025-09-25 05:24:45.512 [info] All environment variables set for unittest execution, PYTHONPATH: "y:\\irenec\\dev\\pipelaunch\\pipelaunch"
2025-09-25 05:24:45.598 [info] Running unittests for workspace y:\irenec\dev\pipelaunch\pipelaunch with arguments: c:\Users\irenec\.vscode\extensions\ms-python.python-2025.14.0-win32-x64\python_files\unittestadapter\execution.py,--udiscovery,-v,-s,src,-p,test_*.py

2025-09-25 05:24:45.598 [info] > Y:\irenec\dev\pipelaunch\venv\Scripts\python.exe ~\.vscode\extensions\ms-python.python-2025.14.0-win32-x64\python_files\unittestadapter\execution.py --udiscovery -v -s src -p test_*.py
2025-09-25 05:24:45.598 [info] cwd: y:\irenec\dev\pipelaunch\pipelaunch

Extension version: 2025.14.0
VS Code version: Code 1.104.1 (0f0d87fa9e96c856c5212fc86db137ac0d783365, 2025-09-17T23:36:24.973Z)
OS version: Windows_NT x64 10.0.26100
Modes:

• Python version (& distribution if applicable, e.g. Anaconda): 3.9.13
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
• Value of the python.languageServer setting: Default

User Settings

Multiroot scenario, following user settings may not apply:

languageServer: "Pylance"

testing
• debugPort: 2000
• unittestArgs: "<placeholder>"
• unittestEnabled: true
• autoTestDiscoverOnSavePattern: "<placeholder>"

Installed Extensions

Extension Name	Extension Id	Version
auto-comment-blocks	kev	1.0.1
autodocstring	njp	0.6.1
copilot	Git	1.372.0
copilot-chat	Git	0.31.2
debugpy	ms-	2025.10.0
extendscript-debug	Ado	2.1.0
gitlens	eam	17.5.1
js-debug	ms-	1.104.0
js-debug-companion	ms-	1.1.3
python	ms-	2025.14.0
vscode-js-profile-table	ms-	1.0.10
vscode-pull-request-github	Git	0.118.1
vscode-pylance	ms-	2025.8.2
vscode-python-envs	ms-	1.8.0

System Info

Item	Value
CPUs	Intel(R) Xeon(R) W-2225 CPU @ 4.10GHz (8 x 4104)
GPU Status	2d_canvas: enabled direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off trees_in_viz: disabled_off video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	undefined
Memory (System)	63.69GB (41.51GB free)
Process Argv	Y:\irenec\vscode\irenec_pipelaunch20.code-workspace --crash-reporter-id 0f3862e7-250e-488d-8961-f1c20f56d1d7
Screen Reader	no
VM	0%

A/B Experiments

vsliv368:30146709
binariesv615:30325510
nativeloc1:31344060
dwcopilot:31170013
dwoutputs:31242946
copilot_t_ci:31333650
e5gg6876:31282496
996jf627:31283433
pythonrdcb7:31342333
usemplatestapi:31297334
aj953862:31281341
cs4_fixed:31391938
nes-set-on:31340697
63221493:31336333
aa_t:31379598
0cj2b977:31352657
cloudbuttont:31379625
todos-0:31390406
v66-short:31391904
control_gpt5applypatchexclusively:31392057
42190218_ostrepl:31382710
metis-embeddings:31388492
trigger-command-fix:31379601
auto_model_enabled:31385282
use-responses-api:31390855
copilot-gpt-5-mini:31391796

[eleanorjboyd]

hm yes I think there is a report on this problem which I am still trying to determine how to get around technically because of how the current flow works. Will investigate and link the existing issue when I find it