Skip to content

Relaxed version enforcement

Jump to bottom
Benny Lutati edited this page Aug 30, 2021 · 4 revisions

ℹ️ The following is a lengthy discussion that you don't have to understand in order to use relaxed-poetry. That said, If you have a poetry background you might find it interesting

There are several changes that were made regarding poetry version enforcement in an attempt to make our life a bit easier.

Python version bounds

As explained in Issue 1413, when installing a new package, poetry checks if the python versions that are given in the pyproject.toml are in full agreement with the python constraints of this package, otherwise it will refuse to install the package. While this behavior have a good reasoning behind it (which you can read about in the referenced issue) it still very confusing and non-intuitive.

To understand the relaxation that is made in relaxed-poetry we can examine the following example:

Assume you just started a new project using the command

poetry new test-project

which resulted in the following pyproject.toml file:

# pyproject.toml

# ...
[tool.poetry.dependencies]
python = "^3.8"

# ...

With a project defined, you try and install numpy

> poetry add numpy

Using version ^1.21.2 for numpy

Updating dependencies
Resolving dependencies... (0.0s)

  SolveFailure

  The current project's Python requirement (>=3.8,<4.0) is not compatible with some of the required packages Python requirement:
    - numpy requires Python >=3.7,<3.11, so it will not be satisfied for Python >=3.11,<4.0
  
  Because no versions of numpy match >1.21.2,<2.0.0
   and numpy (1.21.2) requires Python >=3.7,<3.11, numpy is forbidden.
  So, because test depends on numpy (^1.21.2), version solving failed.

... Removed Exception Information For Conciseness  ...

• Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties
    
    For numpy, a possible solution would be to set the `python` property to ">=3.8,<3.11"

    https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
    https://python-poetry.org/docs/dependency-specification/#using-environment-markers

Although a possible solution is offered on the bottom of the error message, this error is a "bit" confusing. First, it specifies that your project wants Python >=3.8,<4.0 and then that numpy wants python >=3.7,<3.11 - you check your environment and the chosen installation is "3.8" which means that there shouldn't be a problem..

The main reason poetry refuses to install numpy is that if you will publish your new package, a client with python version 3.12 will be able to consume it (since you specified the constraint >=3.8,<4.0). This will obviously not work because your package depends on a numpy that is said not to work for any python version >=3.11.

On the contrast, adding new package in relaxed-poetry validates the package's python-constraints only against the current installed version, so in this case the installation will pass without any problems.

> rp add numpy

Using version ^1.21.2 for numpy

Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing numpy (1.21.2)

But, Poetry's point is still valid, we don't want to publish a package with a too wide python constraint. This why the build command in relaxed-poetry automatically tighten the python constraint based on the dependencies:

> rp build

Building test-project (0.1.0)
Tightening bounds to python version requirements based on dependencies
Resolving dependencies... (0.1s)
Will require python version: >=3.8,<3.11
  - Building sdist
  - Built test-project-0.1.0.tar.gz
  - Building wheel
  - Built test-project-0.1.0-py3-none-any.whl

As can be seen by the output, python version requirements were tighten into >=3.8,<3.11 within the built distributions.

ℹ️ This build command behavior can be disabled using the --keep-python-bounds flag.

Common (though non-standard) python constraints

As can be read in Issue 4201, If a dependent package specifies a required python version constraint using the format: ><=x.y.* (e.g., >=3.6.*) which is not a valid PEP 345/PEP 440 constraint, poetry refuse to install it.

While standards are very important (especially in a chaotic environment like python's eco-system), this constraints are very common and in use by many packages. For this reason, instead of rejecting the dependent package, relaxed-poetry emits a warning and retry the resolution with a truncated constraint (i.e., '>=3.5.*' will be replaced with '>=3.5') in most cases this allows the dependency to be installed.

No enforcement of PEP517

While pep517 have been with us for a long time, there are many new and old packages that does not support it. In Poetry, package installation force the usage of pep517 (via the pip flag --use-pep517) which break the installation of these packages. Relaxed-Poetry changed this so that the default behavior of pip will be taken.

Clone this wiki locally