Tests aren't run for all Python versions anymore

[mgorny]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

See #425 (comment)

I broke something while splitting the feedstock, filing a bug so I don't forget to fix it.

Installed packages

‎

Environment info

‎

[h-vetinari]

Should be good enough to ensure requirements: host: python is present, for pytorch-tests to be built per python version. And we also need to host-depend on {{ pin_subpackage("pytorch") }} in host/run to ensure we don't repackage the pytorch bits in pytorch-tests.

[mgorny]

Thanks for the suggestion.

That said, FWICS pytorch-tests don't include any PyTorch bits — just the test tree. Am I missing something?

[h-vetinari]

That said, FWICS pytorch-tests don't include any PyTorch bits — just the test tree. Am I missing something?

That depends on how the packaging goes. Having {{ pin_subpackage("pytorch") }} in host would ensure those don't get picked up as being "new" from the POV of what it means to install pytorch-tests. And you definitely need to rely on it at runtime, because otherwise pytorch-tests is not functional.

[mgorny]

But isn't the following equivalent to it:

pytorch-cpu-feedstock/recipe/meta.yaml

Lines 362 to 363 in 9f6e138

	- pytorch {{ version }} cuda*_{{ blas_impl }}_py{{ CONDA_PY }}*{{ build }} # [cuda_compiler_version != "None"]
	- pytorch {{ version }} cpu_{{ blas_impl }}_py{{ CONDA_PY }}*{{ build }} # [cuda_compiler_version == "None"]

?

[h-vetinari]

No, test.requires only gets installed during the testing phase. Someone doing mamba install pytorch-tests must also get pytorch along with it. For that it needs to move from test.requires to requirements.run.

[mgorny]

Ahhh. Any other run requirements to include or just that? Presumably copying all test dependencies makes no sense since tests won't be installed as part of the package.

[h-vetinari]

First off, apologies that I didn't review this closely enough in #425.

That said, FWICS pytorch-tests don't include any PyTorch bits — just the test tree. Am I missing something?

When you say "test tree", do you mean that in the same way as "source tree"? Because currently, pytorch-tests is completely empty, whereas my mental model is something like scipy-tests which does contain

lib/python3.14/site-packages/scipy/<foo>/tests/...

Not sure how much effort we want to take to split things, but pytorch currently contains

lib/python3.13/site-packages/torch/testing/...

which should definitely move to pytorch-tests, and stuff like

lib/python3.13/site-packages/torch/_dynamo/__pycache__/test_case.cpython-313.pyc
lib/python3.13/site-packages/torch/_dynamo/__pycache__/test_dont_skip_tracing_functions.cpython-313.pyc
lib/python3.13/site-packages/torch/_dynamo/__pycache__/test_minifier_common.cpython-313.pyc
lib/python3.13/site-packages/torch/_dynamo/__pycache__/testing.cpython-313.pyc

which should probably move too (if it's not too much work and/or too fragile). In scipy we do that granular separation, though we also have build-system support for that (-Cinstall-args=--tags=tests)¹

Ahhh. Any other run requirements to include or just that?

I think once we have a separate output for tests, we should just promote all the dependencies to be actual runtime dependencies.

Footnotes

1. though we did start out with a more manual process, see here ↩

[isuruf]

lib/python3.13/site-packages/torch/testing/...
which should definitely move to pytorch-tests

Those files are part of pytorch and used in downstream projects to define tests and would be a confusing break.

[mgorny]

Yeah, I wouldn't split PyTorch itself. The purpose of pytorch-tests was really to avoid long paths in the test/ tree that's included in the info part, and I wouldn't go beyond that.

[h-vetinari]

Those files are part of pytorch and used in downstream projects to define tests and would be a confusing break.

Ah, thanks for the context, good to know. My comment was purely about files used for internal testing; obviously anything that's used beyond that needs to stay in pytorch proper.

Yeah, I wouldn't split PyTorch itself. The purpose of pytorch-tests was really to avoid long paths in the test/ tree that's included in the info part, and I wouldn't go beyond that.

If the tests can be installed into $SP_DIR without too much hassle (I'm easily swayed that this might not be the case), then I'd like us to package them; even if we keep running only a subset of them in CI, but that would enable interested users to run custom subsets of the test suite themselves (and not just through conda build --test <artefact>, or whatever the invocation is).