Discovering tests with pytest fails if the ".py" extension is removed from _nodeid

[Fieoner]

Type: Bug

Behaviour

Discovering tests with pytest fails if the _nodeid of the test was changed to not contain ".py" during discovery.

Steps to reproduce:

1. Change the _nodeid of a test during discovery:

def pytest_collection_modifyitems(items):
    for item in items:
        item._nodeid = f"some_file.extension::{item.originalname}"

2. Write a test that uses @pytest.mark.parametrize

import pytest

@pytest.mark.parametrize("x", [1, 2])
def test_example(x):
    assert x > 0

You should get the following error:

[error] pytest test discovery error for workspace:  /Users/fieoner/projects/pytest-bug 
 Error Occurred, traceback: Traceback (most recent call last):
  File "/Users/fieoner/.vscode/extensions/ms-python.python-2024.14.1-darwin-arm64/python_files/vscode_pytest/__init__.py", line 389, in pytest_sessionfinish
    session_node: TestNode | None = build_test_tree(session)
  File "/Users/fieoner/.vscode/extensions/ms-python.python-2024.14.1-darwin-arm64/python_files/vscode_pytest/__init__.py", line 460, in build_test_tree
    class_and_method = second_split[1] + "::"  # This has "::" separator at both ends
IndexError: list index out of range

it could be fixed by replacing

class_and_method = second_split[1] + "::"  # This has "::" separator at both ends

with

class_and_method = second_split[1] + "::" if len(second_split) > 1 else ''  # This has "::" separator at both ends

here https://github.com/microsoft/vscode-python/blob/main/python_files/vscode_pytest/__init__.py#L519

Extension version: 2024.14.1
VS Code version: Code 1.93.1 (Universal) (38c31bc77e0dd6ae88a4e9cc93428cc27a56ba40, 2024-09-11T17:20:05.685Z)
OS version: Darwin arm64 23.6.0
Modes:

• Python version (& distribution if applicable, e.g. Anaconda): 3.10.0
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Pyenv
• Value of the python.languageServer setting: Default

User Settings

languageServer: "Pylance"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true

Installed Extensions

Extension Name	Extension Id	Version
Airflow DAG Viewer	WesleyBatista.airflow-dag-viewer	1.0.0
Black Formatter	ms-python.black-formatter	2024.2.0
Code Runner	formulahendry.code-runner	0.12.2
Dev Containers	ms-vscode-remote.remote-containers	0.384.0
Docker	ms-azuretools.vscode-docker	1.29.2
ESLint	dbaeumer.vscode-eslint	3.0.10
GitHub Copilot	GitHub.copilot	1.232.0
GitHub Copilot Chat	GitHub.copilot-chat	0.20.3
GitLens — Git supercharged	eamodio.gitlens	15.5.1
JavaScript Debugger	ms-vscode.js-debug	1.93.0
JavaScript Debugger Companion Extension	ms-vscode.js-debug-companion	1.1.3
Jupyter	ms-toolsai.jupyter	2024.8.1
Jupyter Cell Tags	ms-toolsai.vscode-jupyter-cell-tags	0.1.9
Jupyter Keymap	ms-toolsai.jupyter-keymap	1.1.2
Jupyter Notebook Renderers	ms-toolsai.jupyter-renderers	1.0.19
Jupyter Slide Show	ms-toolsai.vscode-jupyter-slideshow	0.1.6
Makefile Tools	ms-vscode.makefile-tools	0.11.13
Partial Diff	ryu1kn.partial-diff	1.4.3
Prettier - Code formatter	esbenp.prettier-vscode	11.0.0
Pylance	ms-python.vscode-pylance	2024.9.2
Python	ms-python.python	2024.14.1
Python Debugger	ms-python.debugpy	2024.10.0
Python Indent	KevinRose.vsc-python-indent	1.18.0
Rainbow CSV	mechatroner.rainbow-csv	3.12.0
SQL Formatter	adpyke.vscode-sql-formatter	1.4.4
Table Visualizer for JavaScript Profiles	ms-vscode.vscode-js-profile-table	1.0.9
Test Adapter Converter	ms-vscode.test-adapter-converter	0.1.9
Vim	vscodevim.vim	1.28.1

System Info

Item	Value
CPUs	Apple M1 Pro (10 x 2400)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	4, 4, 3
Memory (System)	32.00GB (0.07GB free)
Process Argv	--crash-reporter-id e3b7d057-fedc-4534-bb92-08a0621aa654
Screen Reader	no
VM	0%

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805:30301674
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:31024239
vscaac:30438847
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonnoceb:30805159
asynctok:30898717
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
da93g388:31013173
a69g1124:31058053
dvdeprecation:31068756
dwnewjupytercf:31046870
newcmakeconfigv2:31071590
impr_priority:31102340
nativerepl2:31139839
refactort:31108082
pythonrstrctxt:31112756
flightc:31134773
wkspc-onlycs-t:31132770
wkspc-ranged-t:31125599
ei213698:31121563

[eleanorjboyd]

To clarify you have your own plugin which you use to modify the ids? Could you clarify the use case for changing these Ids? I can look into changing the way we parse the pytest ids into the nested structure but with nested classes it is hard to determine what is a path / file and what isn't

[Fieoner]

I'm using pytest to test some sql queries, so for those tests I use pytest_collection_modifyitems to replace the name of the python file with the name of the sql file. I'm not using another plugin for this, I just have this change to the _nodeid in my conftest.py and then when running the tests I get the sql filenames in the output

[zklaus]

Just to second this and to add another (similar) use-case: I am working on a plugin that generates tests from test definitions in yaml files. The tests are loaded from python files, but parametrized with information from the yaml files, which are also taken to be the path of the tests, which in turn allows us to navigate quickly to the test definition.

This is inspired by the Network Unit Testing System, which might provide another use-case.

[eleanorjboyd]

Sorry for the delay- I will investigate. We seems to have a few plugins which cause edits to tests / test names and how we can try and fix those issues

[github-actions]

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on, then we will be happy to re-open this issue to pick up where we left off.

Happy Coding!

[eleanorjboyd]

Hi! Can you give me an example of the test id and then also the expected nested folder structure from there? I use the :: to created the nested folder structure and not sure if that would then break the structure I build with this custom node id editing.

def pytest_collection_modifyitems(items):
    for item in items:
        item._nodeid = f"some_file.extension::{item.originalname}"