Using only one package manager in project

[ProfessorTom]

I recently made a PR where I addressed the dev container not building out of the box. While doing so, I noticed that we use both pip and poetry when building that container. I believe this pattern of mixing both package managers is prevalent throughout the project e.g. in fly.dockerflie, pyproject.toml and .devcontainer/Dockerfile to list a few.

I asked around at my local Python meetup and was advised that the point of poetry was to have reproducible builds, hence poetry.lock. So, I'd like to propose that one package manager be used.

This suggestion, however, could mean multiple steps.

Why? it turns out that uv is a drop-in replacement for poetry. uv is written in Rust and is quite a bit faster than poetry. In theory we could move to uv in one go, but I'm a cautious man and prefer to do things one step at a time so I can fix what breaks at each step. So Step 1 may be to eliminate pip and only use poetry...which may in turn lead to the very problem that existed with the development container when I submitted my PR: Python 3.8 is no longer supported by several dependencies. So the real Step 1 may be to drop support for Python 3.8 in favor for at least Python 3.9, then remove pip, then move to uv.

Of course, there may be good reason for keeping pip around Maybe there is a dependency that only works with pip (but I'm told that poetry and uv both pull from pypy, so if pip can install the dependency, poetry and uv should be able to do so as well.

I would be more than happy to this work, but it may require a deep understanding of the project. Then again, maybe I could just YOLO poetry or uv only say in development.dockerfile in a branch and see if I can get the project to come up. My concern at that point would be that just having a site that runs may not be a sufficient test.

I welcome any and all thoughts, comments and criticisms.

[JPTomorrow]

I think the big kicker will be dropping python 3.8 support.

uv does seem to have a few quirks that the project would need to consider, such as platform specific requirements files, and the fact that the company behind it (Astral) is a private company. May cause trouble down the line depending on how they want to treat their product.

Getting rid of and deps that rely on pip and rolling them into poetry seems like a good first step though.

[NiklasNeugebauer]

I agree that uv is a separate topic and should not be discussed much further here.

[rodja]

We have an open discussion about dropping Python 3.8 in #4738. The move to a single package manager is a good argument in my opinion. Feel free to create a first shot at this @ProfessorTom.

[ProfessorTom]

A first shot at which: dropping 3.8, moving to poetry or both?

[rodja]

We should probably do both in one PR.

[ProfessorTom]

Do you have any suggestions for testing the move to 3.9 doesn't break anything?

I think there are several places in the code base that are going to need to be updated to support 3.9 as the minimum version number.

On the other hand, maybe I could YOLO it on the basis that release.dockerfile is using 3.11, so there should be backwards compatibility with 3.9, so if the containers come up and don't break or if I can run the project locally, that could be considered sufficient testing.

[rodja]

We also execute the tests via GitHub Actions on all supported Python versions. On your local machine, simply start the devcontainer after adapting .devcontainer/Dockerfile to use 3.9 and run pytest.

[falkoschindler]

Note that there is already a blocked PR about dropping 3.8: #4329.