finalize_distribution_options entrypoint doesn't let me change install_requires

[3N4N]

I am looking for a way to updated the install_requires attribute of distribution. The usecase is to add a specific package as install dependency without putting it in the dependencies section of pyproject.toml.

I have a callable function that hooks into the finalize_distribution_options entrypoint:

def finalize_dist(dist):
    path='pyproject.toml'
    print(" ===================== finalize_dist")
    path = Path(path)
    if not (path.exists() and path.is_file()):
        return

    orig_attr = getattr(dist, 'install_requires')
    print(orig_attr)

    pyproject = toml.load(path)
    install_requires = pyproject.get('project')['dependencies']
    install_requires += [ 'hardcoded-dep1', 'hardcoded-dep2' ]
    print(install_requires)
    setattr(dist, "install_requires", install_requires)

The getattr(dist, 'install_requires') returns None, even though I have an install dependency declared in pyproject.toml ("django>2.1", to be exact). And when I call setattr on dist.install_requires, it doesn't take, i.e., the generated wheel doesn't have "hardcoded-dep1" or "hardcoded-dep2" as install dependency.

Am I completely misunderstanding this?

[3N4N]

This function --

def _dependencies(dist: "Distribution", val: list, _root_dir):
    if getattr(dist, "install_requires", []):
        msg = "`install_requires` overwritten in `pyproject.toml` (dependencies)"
        SetuptoolsWarning.emit(msg)
    _set_config(dist, "install_requires", val)

--makes me think setuptools will overwrite dist.install_requires that I edited from setuptools.finalize_distribution_options entrypoint with the values in pyproject.toml. Is this correct? If it's correct then what I want to do can't be done with finalize_distribution_options entrypoint. What other way is there then?

[3N4N]

_set_config() called from _dependencies() (see the message above) checks for a set_install_requires() in dist.metadata.

def _set_config(dist: "Distribution", field: str, value: Any):
    setter = getattr(dist.metadata, f"set_{field}", None)
    if setter:
        setter(value)
    elif hasattr(dist.metadata, field) or field in SETUPTOOLS_PATCHES:
        setattr(dist.metadata, field, value)
    else:
        setattr(dist, field, value)

I added a callable function that does nothing. So the install dependencies I set from setuptool.finalize_distribution_options entrypoint is kept. That's good enough.

But it seems, to me, to be slightly unintuitive. Why is setuptool.finalize_distribution_options called before pyproject.toml is read? "Finalize" should mean the final changes, no?

And about the setter in dist.metadata: shouldn't dist object be passed to setter as well? What's it gonna do with only the "value"?

[3N4N]

Upon further research I found that I can bind a function to an object:

def finalize_dist(dist):
    # We can set install_requires of dist here, but it's replaced with
    # dependencies from pyproject.toml by setuptools.  Unless we bind this
    # attribute 'set_install_requires' to a function.  And the function needs
    # to be bound to dist object, otherwise it can't update the dist object.
    setattr(dist.metadata,'set_install_requires', MethodType(install_requires_setter, dist))

So, _dependencies() from setuptools will now call install_requires_setter with the dependencies listed in pyproject.toml, and install_requires_setter can add some other packages to that list.

Edit: For completeness, this is what I needed in my pyproject.toml of the setuptools extension:

[project.entry-points."setuptools.finalize_distribution_options"]
setuptools_gitmodules = "setuptools_gitmodules:finalize_dist"

[abravalheri]

Hi @3N4N, please note that, as per PEP 621 a plugin that augments the value of project.dependencies may be doing an invalid operation.

Specifically, the standard says:

...

• A build back-end MUST honour statically-specified metadata (which means the metadata did not list the field in dynamic).
  ...
• Build back-ends MUST raise an error if the metadata specifies a field statically as well as being listed in dynamic.
• If the metadata does not list a field in dynamic, then a build back-end CANNOT fill in the requisite metadata on behalf of the user (i.e. dynamic is the only way to allow a tool to fill in metadata and the user must opt into the filling in).

...

The TL;DR is that if you want to dynamically add dependencies, the pyproject.toml of the project using your plugin needs to specify project.dynamic = ["dependencies"]. However, if they do so, they cannot set project.dependencies.

[3N4N]

Got it. Thank you, @abravalheri.