8.2.0 Breaking builds with Python3.8 Images

[solanoam]

Hey team 👋

It was a not so happy Sunday for us at our company as we started our week with dreaded message:
ModuleNotFoundError: No module named 'setuptools.command.build'

Working with a big project, with Python 3.8, which is bundled with old setuptools, we did not have issues prior to this version, but builds started to fail after this release.

the only fix was to manually run through every docker image that is on python 3.8, and upgrade the setuptools.

Do you guys have any other suggestions to manage this issue? setuptools must be upgraded?

if this is the case, i strongly suggest to change this version to a major, so it won't break builds unintentionally for dependencies that were already working fine before.

More technical details:

failure when running pytest:

Traceback (most recent call last):

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/bin/pytest", line 8, in <module>

[2025-02-23T18:42:58.785Z] test_admin_1  |     sys.exit(main())

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/_pytest/config/__init__.py", line 65, in main

[2025-02-23T18:42:58.785Z] test_admin_1  |     config = _prepareconfig(args, plugins)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/_pytest/config/__init__.py", line 213, in _prepareconfig

[2025-02-23T18:42:58.785Z] test_admin_1  |     return pluginmanager.hook.pytest_cmdline_parse(

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__

[2025-02-23T18:42:58.785Z] test_admin_1  |     return self._hookexec(self, self.get_hookimpls(), kwargs)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec

[2025-02-23T18:42:58.785Z] test_admin_1  |     return self._inner_hookexec(hook, methods, kwargs)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>

[2025-02-23T18:42:58.785Z] test_admin_1  |     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall

[2025-02-23T18:42:58.785Z] test_admin_1  |     gen.send(outcome)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/_pytest/helpconfig.py", line 94, in pytest_cmdline_parse

[2025-02-23T18:42:58.785Z] test_admin_1  |     config = outcome.get_result()

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result

[2025-02-23T18:42:58.785Z] test_admin_1  |     raise ex[1].with_traceback(ex[2])

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall

[2025-02-23T18:42:58.785Z] test_admin_1  |     res = hook_impl.function(*args)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/_pytest/config/__init__.py", line 789, in pytest_cmdline_parse

[2025-02-23T18:42:58.785Z] test_admin_1  |     self.parse(args)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/_pytest/config/__init__.py", line 997, in parse

[2025-02-23T18:42:58.785Z] test_admin_1  |     self._preparse(args, addopts=addopts)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/_pytest/config/__init__.py", line 943, in _preparse

[2025-02-23T18:42:58.785Z] test_admin_1  |     self.pluginmanager.load_setuptools_entrypoints("pytest11")

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/manager.py", line 300, in load_setuptools_entrypoints

[2025-02-23T18:42:58.785Z] test_admin_1  |     self.register(plugin, name=ep.name)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/_pytest/config/__init__.py", line 328, in register

[2025-02-23T18:42:58.785Z] test_admin_1  |     ret = super(PytestPluginManager, self).register(plugin, name)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/manager.py", line 127, in register

[2025-02-23T18:42:58.785Z] test_admin_1  |     hook._maybe_apply_history(hookimpl)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/hooks.py", line 333, in _maybe_apply_history

[2025-02-23T18:42:58.785Z] test_admin_1  |     res = self._hookexec(self, [method], kwargs)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec

[2025-02-23T18:42:58.785Z] test_admin_1  |     return self._inner_hookexec(hook, methods, kwargs)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>

[2025-02-23T18:42:58.785Z] test_admin_1  |     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall

[2025-02-23T18:42:58.785Z] test_admin_1  |     return outcome.get_result()

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result

[2025-02-23T18:42:58.785Z] test_admin_1  |     raise ex[1].with_traceback(ex[2])

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall

[2025-02-23T18:42:58.785Z] test_admin_1  |     res = hook_impl.function(*args)

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/setuptools/_vendor/typeguard/_pytest_plugin.py", line 53, in pytest_addoption

[2025-02-23T18:42:58.785Z] test_admin_1  |     add_ini_option("string")

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/setuptools/_vendor/typeguard/_pytest_plugin.py", line 22, in add_ini_option

[2025-02-23T18:42:58.785Z] test_admin_1  |     parser.addini(

[2025-02-23T18:42:58.785Z] test_admin_1  |   File "/placer/server_env/lib/python3.8/site-packages/_pytest/config/argparsing.py", line 131, in addini

[2025-02-23T18:42:58.785Z] test_admin_1  |     assert type in (None, "pathlist", "args", "linelist", "bool")

failure when installing requirements:

[2025-02-23T18:54:27.479Z] #23 [17/21] RUN cd placer && pip install -q -r /placer/bnguru/eset/requirements.txt

[2025-02-23T18:54:27.479Z] #23 sha256:02f8011c718df93cfe137c4294140e1d851db13f5d827cc6f1ce475484961ff7

[2025-02-23T18:55:06.188Z] #23 38.48   error: subprocess-exited-with-error

[2025-02-23T18:55:06.188Z] #23 38.48   

[2025-02-23T18:55:06.188Z] #23 38.48   × python setup.py egg_info did not run successfully.

[2025-02-23T18:55:06.188Z] #23 38.48   │ exit code: 1

[2025-02-23T18:55:06.188Z] #23 38.48   ╰─> [70 lines of output]

[2025-02-23T18:55:06.188Z] #23 38.48       /usr/local/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'tests_require'

[2025-02-23T18:55:06.188Z] #23 38.48         warnings.warn(msg)

[2025-02-23T18:55:06.188Z] #23 38.48       /tmp/pip-install-vay25l0w/lz4_cc31399eba0141d18a51a3292a431833/.eggs/setuptools_scm-8.2.0-py3.8.egg/setuptools_scm/_integration/setuptools.py:31: RuntimeWarning:

[2025-02-23T18:55:06.188Z] #23 38.48       ERROR: setuptools==57.5.0 is used in combination with setuptools-scm>=8.x

[2025-02-23T18:55:06.188Z] #23 38.48       

[2025-02-23T18:55:06.188Z] #23 38.48       Your build configuration is incomplete and previously worked by accident!

[2025-02-23T18:55:06.188Z] #23 38.48       setuptools-scm requires setuptools>=61

[2025-02-23T18:55:06.188Z] #23 38.48       

[2025-02-23T18:55:06.188Z] #23 38.48       Suggested workaround if applicable:

[2025-02-23T18:55:06.188Z] #23 38.48        - migrating from the deprecated setup_requires mechanism to pep517/518

[2025-02-23T18:55:06.188Z] #23 38.48          and using a pyproject.toml to declare build dependencies

[2025-02-23T18:55:06.188Z] #23 38.48          which are reliably pre-installed before running the build tools

[2025-02-23T18:55:06.188Z] #23 38.48       

[2025-02-23T18:55:06.188Z] #23 38.48         warnings.warn(

[2025-02-23T18:55:06.188Z] #23 38.48       WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'

[2025-02-23T18:55:06.188Z] #23 38.48       Traceback (most recent call last):

[2025-02-23T18:55:06.188Z] #23 38.48         File "/tmp/pip-install-vay25l0w/lz4_cc31399eba0141d18a51a3292a431833/.eggs/setuptools_scm-8.2.0-py3.8.egg/setuptools_scm/_integration/pyproject_reading.py", line 36, in read_pyproject

[2025-02-23T18:55:06.188Z] #23 38.48           section = defn.get("tool", {})[tool_name]

[2025-02-23T18:55:06.188Z] #23 38.48       KeyError: 'setuptools_scm'

[2025-02-23T18:55:06.188Z] #23 38.48       running egg_info

[2025-02-23T18:55:06.188Z] #23 38.48       creating /tmp/pip-pip-egg-info-m6hodig6/lz4.egg-info

[2025-02-23T18:55:06.188Z] #23 38.48       writing /tmp/pip-pip-egg-info-m6hodig6/lz4.egg-info/PKG-INFO

[2025-02-23T18:55:06.188Z] #23 38.48       writing dependency_links to /tmp/pip-pip-egg-info-m6hodig6/lz4.egg-info/dependency_links.txt

[2025-02-23T18:55:06.188Z] #23 38.48       writing requirements to /tmp/pip-pip-egg-info-m6hodig6/lz4.egg-info/requires.txt

[2025-02-23T18:55:06.188Z] #23 38.48       writing top-level names to /tmp/pip-pip-egg-info-m6hodig6/lz4.egg-info/top_level.txt

[2025-02-23T18:55:06.188Z] #23 38.48       writing manifest file '/tmp/pip-pip-egg-info-m6hodig6/lz4.egg-info/SOURCES.txt'

[2025-02-23T18:55:06.188Z] #23 38.48       Traceback (most recent call last):

[2025-02-23T18:55:06.188Z] #23 38.48         File "<string>", line 2, in <module>

[2025-02-23T18:55:06.188Z] #23 38.48         File "<pip-setuptools-caller>", line 34, in <module>

[2025-02-23T18:55:06.188Z] #23 38.48         File "/tmp/pip-install-vay25l0w/lz4_cc31399eba0141d18a51a3292a431833/setup.py", line 157, in <module>

[2025-02-23T18:55:06.188Z] #23 38.48           setup(

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup

[2025-02-23T18:55:06.188Z] #23 38.48           return distutils.core.setup(**attrs)

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/distutils/core.py", line 148, in setup

[2025-02-23T18:55:06.188Z] #23 38.48           dist.run_commands()

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/distutils/dist.py", line 966, in run_commands

[2025-02-23T18:55:06.188Z] #23 38.48           self.run_command(cmd)

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command

[2025-02-23T18:55:06.188Z] #23 38.48           cmd_obj.run()

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 299, in run

[2025-02-23T18:55:06.188Z] #23 38.48           self.find_sources()

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 306, in find_sources

[2025-02-23T18:55:06.188Z] #23 38.48           mm.run()

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 541, in run

[2025-02-23T18:55:06.188Z] #23 38.48           self.add_defaults()

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 578, in add_defaults

[2025-02-23T18:55:06.188Z] #23 38.48           sdist.add_defaults(self)

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/distutils/command/sdist.py", line 226, in add_defaults

[2025-02-23T18:55:06.188Z] #23 38.48           self._add_defaults_python()

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/site-packages/setuptools/command/sdist.py", line 107, in _add_defaults_python

[2025-02-23T18:55:06.188Z] #23 38.48           build_py = self.get_finalized_command('build_py')

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/distutils/cmd.py", line 299, in get_finalized_command

[2025-02-23T18:55:06.188Z] #23 38.48           cmd_obj.ensure_finalized()

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/distutils/cmd.py", line 107, in ensure_finalized

[2025-02-23T18:55:06.188Z] #23 38.48           self.finalize_options()

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/site-packages/setuptools/command/build_py.py", line 37, in finalize_options

[2025-02-23T18:55:06.188Z] #23 38.48           orig.build_py.finalize_options(self)

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/distutils/command/build_py.py", line 43, in finalize_options

[2025-02-23T18:55:06.188Z] #23 38.48           self.set_undefined_options('build',

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/distutils/cmd.py", line 286, in set_undefined_options

[2025-02-23T18:55:06.188Z] #23 38.48           src_cmd_obj = self.distribution.get_command_obj(src_cmd)

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/distutils/dist.py", line 857, in get_command_obj

[2025-02-23T18:55:06.188Z] #23 38.48           klass = self.get_command_class(command)

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/site-packages/setuptools/dist.py", line 854, in get_command_class

[2025-02-23T18:55:06.188Z] #23 38.48           self.cmdclass[command] = cmdclass = ep.load()

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2450, in load

[2025-02-23T18:55:06.188Z] #23 38.48           return self.resolve()

[2025-02-23T18:55:06.188Z] #23 38.48         File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2456, in resolve

[2025-02-23T18:55:06.188Z] #23 38.48           module = __import__(self.module_name, fromlist=['__name__'], level=0)

[2025-02-23T18:55:06.188Z] #23 38.48       ModuleNotFoundError: No module named 'setuptools.command.build'

[2025-02-23T18:55:06.188Z] #23 38.48       [end of output]

[2025-02-23T18:55:06.188Z] #23 38.48   

[2025-02-23T18:55:06.188Z] #23 38.48   note: This error originates from a subprocess, and is likely not a problem with pip.

[2025-02-23T18:55:06.188Z] #23 38.48 error: metadata-generation-failed

[2025-02-23T18:55:06.188Z] #23 38.48 

[2025-02-23T18:55:06.188Z] #23 38.48 × Encountered error while generating package metadata.

[2025-02-23T18:55:06.188Z] #23 38.48 ╰─> See above for output.

[2025-02-23T18:55:06.188Z] #23 38.48 

[2025-02-23T18:55:06.188Z] #23 38.48 note: This is an issue with the package mentioned above, not pip.

[2025-02-23T18:55:06.188Z] #23 38.48 hint: See above for details.

[2025-02-23T18:55:06.188Z] #23 38.49 

[2025-02-23T18:55:06.188Z] #23 38.49 [notice] A new release of pip is available: 23.0.1 -> 25.0.1

[2025-02-23T18:55:06.188Z] #23 38.49 [notice] To update, run: pip install --upgrade pip

[2025-02-23T18:55:06.754Z] #23 ERROR: process "/bin/sh -c cd placer && pip install -q -r /placer/bnguru/eset/requirements.txt" did not complete 

[abhinavkaul95]

Hi team, we are also blocked because we are using python-measurement library which internally uses this library. Please resolve this issue as soon as possible.

[RonnyPfannschmidt]

its unclear whats happening there - what version of setuptools an pip is involved - it seems like a non-standard build process is used

is there a minimal reproducer i could run?

[RonnyPfannschmidt]

this version lists setuptools 61 as a dependency and its unclear whats happening on your end - it seems like something is invoked wrong and installed wrong

[RonnyPfannschmidt]

the error/warning triggered at the beginning tells something important - the setup was at best working by accident only
its unclear to me why the setuptools import fails now with the minimal versions - i strongly suspect a miss-configured build process as package/wheel builds should be isolated and follow the peps

[solanoam]

That may be true, but the fact remains: a dependency we rely on suddenly started failing during builds due to a new version. This is a textbook example of a breaking change.
I’ve also noticed that other developers are reporting similar issues.

I would consider marking this version as such.

[solanoam]

its unclear whats happening there - what version of setuptools an pip is involved - it seems like a non-standard build process is used

is there a minimal reproducer i could run?

I will try to work something out

[RonnyPfannschmidt]

based on the information i have this is currently solely affecting a eol python version in potentially unsupported build setups
unless this is affecting demonstrable correct build setups it ought not to be considered a breaking change

its not intentionally braking anything
its not having necessary migration steps

[RonnyPfannschmidt]

for the initial version ported - the fact that a egg of setuptools_scm is used and that there is a attempt to build a egg of lz4 points to a completely broken build setup and setuptools exploding itself

[Pebin]

I will just add that it's not strictly connected just to python 3.8, it crashed even our pipelines and we are on 3.9

[RonnyPfannschmidt]

The initial happens to brokenly build a compression library that actually provides wheels

So please provide reproducers not just logs if broken builds

[RonnyPfannschmidt]

https://xkcd.com/1172/ feels relevant

[solanoam]

https://xkcd.com/1172/ feels relevant

lol 😂

at least in the meme it’s a separate cpu, so the user has a choice to buy the old model 🙃