Python Pytest parametrized test with dynamic name causes test running to fail

[pzelnip]

Type: Bug

Behaviour

If you have a Pytest parametrized test who's identifier is generated dynamically, then running tests from the test panel in VS Code fails with an error.

Steps to reproduce:

1. Use this test:

import uuid


def make_uuid():
    return str(uuid.uuid4())


@pytest.mark.parametrize(
    "version_id",
    [
        make_uuid(),
        "0186e255-1da4-d860-d556-504ab56eff33",
    ],
)
def test_show_bug(version_id):
    pass

If you run this test, it fails with an error similar to:

Running pytest with args: ['-p', 'vscode_pytest', '--rootdir=/Users/adamparkin/temp/sandbox/project', '/Users/adamparkin/temp/sandbox/project/backend/project/test_vscode.py::test_show_bug[6e6e8a02-2afd-49e2-b770-6c5b0a4c6508]', '/Users/adamparkin/temp/sandbox/project/backend/project/test_vscode.py::test_show_bug[0186e255-1da4-d860-d556-504ab56eff33]']
============================= test session starts ==============================
platform darwin -- Python 3.11.6, pytest-8.3.4, pluggy-1.5.0
cachedir: .cache
django: version: 4.2.17, settings: config.settings.test (from env)
rootdir: /Users/adamparkin/temp/sandbox/project
configfile: backend/pytest.ini
plugins: cov-6.0.0, random-0.2, pudb-0.7.0, sugar-1.0.0, env-1.1.5, ddtrace-2.17.2, django-4.9.0, anyio-4.7.0, mock-3.14.0
collected 1 item

--- generated xml file: /Users/adamparkin/temp/sandbox/project/junit.xml ----
============================ no tests ran in 0.02s =============================
ERROR: not found: /Users/adamparkin/temp/sandbox/project/backend/project/test_vscode.py::test_show_bug[6e6e8a02-2afd-49e2-b770-6c5b0a4c6508]
(no match in any of [<Module test_vscode.py>])

Finished running tests!

If you comment out the call to make_uuid in the parameterization block it succeeds without issue.

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

VS Code version: Code 1.96.0 (138f619c86f1199955d53b4166bef66ef252935c, 2024-12-11T02:29:09.626Z)
OS version: Darwin arm64 22.6.0
Modes:

System Info

Item	Value
CPUs	Apple M1 Pro (10 x 2400)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	6, 8, 9
Memory (System)	32.00GB (0.13GB free)
Process Argv	--crash-reporter-id c9273746-3cb8-448c-9201-064e551e6d2f
Screen Reader	no
VM	0%

Extensions (91)

Extension	Author (truncated)	Version
better-comments	aar	3.0.2
vscode-base64	ada	0.1.0
codesnap	adp	1.3.4
vscode-sql-formatter	adp	1.4.4
vscode-sqlite	ale	0.14.1
atlascode	atl	3.0.15
django-html	bib	1.3.0
emojisense	bie	0.10.0
ruff	cha	2024.56.0
vscode-postgres	cko	1.4.3
vscode-markdownlint	Dav	0.57.0
vscode-eslint	dba	3.0.10
vscode-faker	dee	2.0.0
python-environment-manager	don	1.2.4
vscode-babel-coloring	dza	0.0.4
gitlens	eam	16.0.5
EditorConfig	Edi	0.16.4
prettier-vscode	esb	11.0.0
hadolint	exi	1.1.2
url-encode	fle	1.1.0
auto-rename-tag	for	0.1.10
code-runner	for	0.12.2
seito-openfile	Fr4	2.2.3
copilot	Git	1.252.0
copilot-chat	Git	0.23.1
vscode-pull-request-github	Git	0.102.0
gitlab-workflow	Git	5.23.1
gc-excelviewer	Gra	4.2.62
todo-tree	Gru	0.0.226
timing	Haa	2.7.2
terraform	has	2.34.1
yaml-plus-json	hil	1.12.2
vscode-htmlhint	HTM	1.0.5
rest-client	hum	0.25.1
copy-markdown-as-html	jer	1.1.0
jwt-decoder	jfl	1.2.0
vscode-random	jre	1.12.0
tasks-context	jsa	1.0.0
presentationmode	jsp	1.1.5
vscode-gutter-preview	kis	0.32.2
vscode-view-in-browser	kop	1.0.5
markdown-shortcuts	mdi	0.12.0
rainbow-csv	mec	3.13.0
git-graph	mhu	1.30.0
djlint	mon	2024.10.0
vscode-docker	ms-	1.29.3
black-formatter	ms-	2024.4.0
debugpy	ms-	2024.14.0
flake8	ms-	2023.10.0
isort	ms-	2023.10.1
pylint	ms-	2024.0.0
python	ms-	2024.22.0
vscode-pylance	ms-	2024.12.1
jupyter	ms-	2024.11.0
jupyter-keymap	ms-	1.1.2
jupyter-renderers	ms-	1.0.21
vscode-jupyter-cell-tags	ms-	0.1.9
vscode-jupyter-slideshow	ms-	0.1.6
remote-containers	ms-	0.395.0
remote-ssh	ms-	0.116.0
remote-ssh-edit	ms-	0.87.0
remote-wsl	ms-	0.88.5
vscode-remote-extensionpack	ms-	0.26.0
hexeditor	ms-	1.11.1
live-server	ms-	0.4.15
remote-explorer	ms-	0.4.3
remote-server	ms-	1.5.2
sublime-keybindings	ms-	4.1.10
autodocstring	njp	0.6.1
kitty-time	QzS	0.5.2
vscode-yaml	red	1.15.0
vscode-coverage-gutters	rya	2.12.0
partial-diff	ryu	1.4.3
jinjahtml	sam	0.20.0
vscode-javascript-booster	sbu	14.0.1
jenkinsfile-support	sec	0.1.0
vscode-spotify	shy	3.2.1
svg-preview	Sim	2.8.3
vscode-fileutils	sle	3.10.3
sourcery	sou	1.27.0
rewrap	stk	1.16.3
avro	str	0.5.0
code-spell-checker	str	4.0.21
even-better-toml	tam	0.19.2
vscode-djaneiro	the	1.4.2
mako	tom	0.2.0
vscode-pets	ton	1.27.0
vscode-icons	vsc	12.9.0
change-case	wma	1.0.0
markdown-all-in-one	yzh	3.6.2
html-css-class-completion	Zig	1.20.0

(1 theme extensions excluded)

A/B Experiments

vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805:30301674
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythonnoceb:30805159
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
dvdeprecation:31068756
dwnewjupytercf:31046870
2f103344:31071589
nativerepl1:31139838
pythonrstrctxt:31112756
nativeloc2:31192216
cf971741:31144450
iacca1:31171482
notype1:31157159
5fd0e150:31155592
dwcopilot:31170013
stablechunks:31184530
6074i472:31201624

[karthiknadig]

The issue is that the test id changes for each run. In cases like these there is no way for us to discover and run. Basically, these two runs results in different ids.

> pytest --collect-only -vvv
> pytest -vvv

In these cases you won't be able to individual select and run even using the pytest CLI, you will have to run the whole thing.

Bug

It should work when you run the whole thing. Currently, we ask pytest to run the entire set of tests by passing the testcases by id. We could change it to detect the common parent and pass the common parent. /cc @eleanorjboyd

Workaround

If you provide a id function or include id in your params it should work as expected: https://docs.pytest.org/en/7.1.x/example/parametrize.html#different-options-for-test-ids

import uuid
import pytest

def make_uuid():
    return str(uuid.uuid4())


@pytest.mark.parametrize(
    "version_id",
    [
        pytest.param(make_uuid(), id='1'),
        pytest.param("0186e255-1da4-d860-d556-504ab56eff33", id='2'),
    ],
)
def test_show_bug(version_id):
    pass

[pzelnip]

Cheers, that workaround works, if I supply an id value to the dynamic parametrized case, test executes fine:

@pytest.mark.parametrize(
    "version_id",
    [
        pytest.param(make_uuid(), id="random uuid"),
        "0186e255-1da4-d860-d556-504ab56eff33",
    ],
)
def test_show_bug(version_id):
    pass

[eleanorjboyd]

yeah the workaround is your best bet- I answer why this wasn't possible from a technical direction here: #22648