Setuptools 9.1.0 break xmlsec and likely other installation (new issue - not the same as in 9.0.*)

[potiuk]

It looks like setuptools-scm 9.1.0 released 2 hours ago again breaks xmlsec installation from sources. The errors we start gettiing in Airflow CI indicate that the source package has 0.0.0 version that does not match declared version, and timing of the event indicate that this is related to the 9.1.0 release of setuptools-scm - even if it is not obvious from errors.

This affects both pip install and uv pip install and multiple versions of xmlsec are affected:

xmlsec/python-xmlsec#369 (pip install):

WARNING: Requested xmlsec==1.3.13 from https://files.pythonhosted.org/packages/.../xmlsec-1.3.13.tar.gz (from -r requirements.txt (line 7)), but installing version 0.0.0
Discarding ... (requires-python:>=3.5): Requested xmlsec==1.3.13 ... has inconsistent version: expected '1.3.13', but metadata has '0.0.0'

Airflow CI failures (uv pip install):

https://github.com/apache/airflow/actions/runs/16876946675/job/47804260895?pr=54351#step:6:1153

  #45 238.2   × Failed to download and build `xmlsec==1.3.16`
  #45 238.2   ╰─▶ Package metadata version `0.0.0` does not match given version `1.3.16`
  #45 238.2   help: `xmlsec` (v1.3.16) was included because
  #45 238.2         `apache-airflow-providers-amazon[python3-saml]` (v9.12.0) depends
  #45 238.2         on `xmlsec`
  #45 ERROR: process "/bin/bash -o pipefail -o errexit -o nounset -o nolog -c bash /scripts/docker/install_airflow_when_building_images.sh" did not complete successfully: exit code: 1
  ------
   > [main 16/24] RUN --mount=type=cache,id=ci-amd64-0,target=/root/.cache/ bash /scripts/docker/install_airflow_when_building_images.sh:
  231.4    Building mysqlclient==2.2.7
  231.4    Building krb5==0.7.1
  231.7    Building lxml==6.0.0
  232.6       Built pure-sasl==0.6.2
  
  238.2   × Failed to download and build `xmlsec==1.3.16`
  238.2   ╰─▶ Package metadata version `0.0.0` does not match given version `1.3.16`
  238.2   help: `xmlsec` (v1.3.16) was included because
  238.2         `apache-airflow-providers-amazon[python3-saml]` (v9.12.0) depends
  238.2         on `xmlsec`

The way to reproduce it:

pip install --no-binary xmlsec xmlsec==1.3.16

or

uv pip install --no-binary xmlsec xmlsec==1.3.16

Same with 1.3.14 as well.

[RonnyPfannschmidt]

release yanked - there seems to be a edge case with setuptools versions and the incorrect/partial project metadata in xmlsec - i'll add a regression test for the particular package

[potiuk]

Thanks for such quick reaction! ❤️

[gopidesupavan]

cool thank you ..

[joshsny]

Thank you for such a quick response to this 🙏

[alexdutton]

I don't know if it's the same issue, but py_env-python3/bin/python -m pip install . in a clone of black==21.12b0 (as invoked by the environment initialization of pre-commit) also fails to build _black_version.py with 9.1.0, whereas it worked with the previous stable release (8.3.1).

This can be reproduced using:

$ git clone git@github.com:psf/black.git --branch 21.12b0
$ cd black
$ python -m venv venv
$ venv/bin/pip install setuptools_scm[toml]==9.1.0 click==8.0.2  # older black requires an older click too
$ venv/bin/pip install . --no-build-isolation
$ venv/bin/black
Traceback (most recent call last):
  File ".../black/venv/bin/black", line 5, in <module>
    from black import patched_main
  File ".../black/venv/lib/python3.12/site-packages/black/__init__.py", line 39, in <module>
    from black.nodes import STARS, syms, is_simple_decorator_expression
  File ".../black/venv/lib/python3.12/site-packages/black/nodes.py", line 30, in <module>
    from black.cache import CACHE_DIR
  File ".../black/venv/lib/python3.12/site-packages/black/cache.py", line 13, in <module>
    from _black_version import version as __version__
ModuleNotFoundError: No module named '_black_version'

Obviously that is a ridiculously old version of black, but it is another example of the split pyproject.toml/setup.py approach also referenced at #1191 (comment).

Edit: Can report that #1195 resolves this issue with building black ✔️

[zanieb]

It'd be great to add a note to the GitHub Release too please!

[zanieb]

This also affects ujson https://github.com/astral-sh/ruff/actions/runs/16877740131/job/47806334320?pr=19669

  WARNING: Requested ujson==5.7.0 from https://files.pythonhosted.org/packages/43/1a/b0a027144aa5c8f4ea654f4afdd634578b450807bb70b9f8bad00d6f6d3c/ujson-5.7.0.tar.gz (from -r requirements-dev.txt (line 199)), but installing version 0.0.0
Discarding https://files.pythonhosted.org/packages/43/1a/b0a027144aa5c8f4ea654f4afdd634578b450807bb70b9f8bad00d6f6d3c/ujson-5.7.0.tar.gz (from https://pypi.org/simple/ujson/) (requires-python:>=3.7): Requested ujson==5.7.0 from https://files.pythonhosted.org/packages/43/1a/b0a027144aa5c8f4ea654f4afdd634578b450807bb70b9f8bad00d6f6d3c/ujson-5.7.0.tar.gz (from -r requirements-dev.txt (line 199)) has inconsistent version: expected '5.7.0', but metadata has '0.0.0'
ERROR: Could not find a version that satisfies the requirement ujson==5.7.0 (from versions: 1.4, 1.6, 1.8, 1.9, 1.15, 1.18, 1.19, 1.21, 1.22, 1.23, 1.30, 1.33, 1.34, 1.35, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 3.0.0, 3.1.0, 3.2.0, 4.0.0, 4.0.1, 4.0.2, 4.1.0, 4.2.0, 4.3.0, 5.0.0, 5.1.0, 5.2.0, 5.3.0, 5.4.0, 5.5.0, 5.6.0, 5.7.0, 5.8.0, 5.9.0, 5.10.0)

Could you please generalize the yank message? It's pretty confusing right now as "regression in a edge case with xmlsec". I think it should be something like "Package versions unintentionally resolve to 0.0.0 breaking installation"

[RonnyPfannschmidt]

i updated the yank message

[RonnyPfannschmidt]

root cause found - there was a condition check missing for the specific case where setuptools-scm is in build requires, dynamic is not set as the project section is missing and the tool.setuptools_scm section is missing in toml

its however still a valid call as use_scm_version=True is in setup.py

i added a unittest for the case and will keep the regression test with xmlspec as that particular version is a nice reminder of incomplete project setups

[potiuk]

BTW. I think we should all applaud @notatallshaw and gently ask pip team to quickly release the upcoming --build-constraints flag of pip coming in pypa/pip#13534 -> (following similar --build-constraints flag in uv). Once we have it for all major installers out there, it will be infinitely easier to deal with those accidental breakages - both in CI and when - possibly - providing the users with workarounds on how to install the library in case of similar issues.

I personally plan to expand our "reproducible installation" instructions for Airflow with adding --build-constraints flag and freezing "working version" of build dependencies for each airflow version (we already to that automatically for regular dependencies) - and I hope this will also work well in the future with the lock files, though current PEP-751 version of the lock files had it removed unfortunately https://peps.python.org/pep-0751/#locking-build-requirements-for-sdists

[RonnyPfannschmidt]

i absolutely love that more pinning is coming and i absolutely dread what it will do to some feedback loops - i hope we also get matching automation to get feedback timely