Ability to skip locking/pinning of dependency version

[fredrikaverpil]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Feature Request

What?

Ability to add "unlocked" or "unpinned" dependencies.

Why?

I've got loads of small library and application projects which all have few production dependencies I wish to lock the version for. But then I usually have a large number of developer dependencies. When using Poetry to manage this, I wish I had a nice solution to define that I don't need poetry to lock certain (often dev-) dependencies.

I know I'm not alone in experiencing the churn/fatigue with dependabot and opting for e.g. setuptools, flit, hatch et al instead of poetry only to make the maintenance burden easier and rely on failing CI builds/test if something breaks. Here are just some examples of fairly popular projects which are not pinning all of their dependencies to one specific version to make the case that this is not a niche request or use case:

• django
• black
• structlog
• briefcase

Other reasons/motivations:

• I have an internal project meant (by design) to be installed always using its latest versions as it distributes schemas within our organization. Using Poetry's ability to add this project as git link is excellent. But having to update the lockfile in all sorts of projects is cumbersome.
• For some projects, it makes more sense to rely on CI/builds to fail and then go back and pin certain dependencies rather than locking/pinning everything by default.
• It is said that poetry is meant for applications primarily, and not so much libraries. Wouldn't poetry be an amazing fit for libraries if it was possible to opt in to have certain dependencies un-pinned?

Why not use workaround...

Another CI bot with feature X

Because I like Poetry, and I don't want to add a bandaid on top of it. I just wish I was able to make an informed decision myself here.

A side-car requirements.dev.txt

That would make the developer experience less great. And how would I know that a poetry-locked dependency isn't overwritten if I perform another dependency installation outside of the Poetry resolver?
I would be better off looking into setuptools/flit/hatch and pip-tools. But then I would personally miss some very nice features in Poetry.

Ideas

How about:

[tool.poetry.dev-dependencies]
pytest = { version: ">=7.1.1", lock: false }
black = { version: "*", lock: false }
flake8 = { lock: false }

• If another dependency depends on e.g. pytest (in the above example), and it has pytest pinned to a certain version, this pinning will be respected.
• poetry export would export the unpinned dependencies, as defined in pyproject.toml.
• If plugins could hook into the locking/resolver, maybe a plugin could be written to enable this?

[dimbleby]

I'm struggling to understand either the use case or the proposal. I think what you're asking for is that poetry does the usual solving, but that you want some parts of the solution that it finds not to be recorded in the lockfile - is that right? And the reason is that... dependabot is annoying?

I don't think this is a good idea - IMO it's quite contrary to what poetry is for, repeatable builds and locked dependencies are core goals and if that's not what you want then you probably don't want poetry.

But even if it is a good idea, I'm pretty sure that it's far more work than anyone is likely to think worthwhile

• for every dependency poetry would have to track whether that dependency was added for a reason that you did want recording, or for a reason that you didn't want recording.
• eg the whole way that poetry install works is by reading the lockfile: but you want the lockfile not to contain those answers any more

... this isn't just a "sure, let's add that option" kind of thing.

Using poetry to install the dependencies that you do want locking and something else eg pip to install the dependencies that you don't want locking makes much more sense to me - pick the right tool for the job.

[fredrikaverpil]

I'm struggling to understand either the use case or the proposal. I think what you're asking for is that poetry does the usual solving, but that you want some parts of the solution that it finds not to be recorded in the lockfile - is that right?

Yes, correct!

And the reason is that... dependabot is annoying?

Well, not only dependabot - but managing the dependencies you don't intend to keep pinned, which are primarily development dependencies. Imagine maintaining some ~10 repos where each project is semi-actively developed and when dependency X comes out in a new version you want to be on that - in all your projects. This requires quite an amount of energy to cope with, especially if you are the only developer maintaining. I just looked and I average 30 developer dependencies in my [tool.poetry.dev-dependencies] section.

But even if it is a good idea, I'm pretty sure that it's far more work than anyone is likely to think worthwhile

That's totally understandable and I would be the first to respect this. If this is the general consensus among owners/maintainers, then please go ahead and close this issue. Setting the scope of your project is important, but in this case - I think it would bring a lot of value.

Using poetry to install the dependencies that you do want locking and something else eg pip to install the dependencies that you don't want locking makes much more sense to me

This makes no sense to me. How do I know pip is not overwriting something I do want to be locked by Poetry?
This entire feature request/issue is about making Poetry the tool of choice to manage my dependencies; not adding some other tool on top.

[dimbleby]

How do I know pip is not overwriting something I do want to be locked by Poetry?

run pip first and poetry second

[neersighted]

I'm going to close this as unplanned, for now. Poetry's fundamental design is to install from the lock file.

We are working on designs for installing non-importable tools (e.g. tox -- pytest is importable and thus we think it should be locked) separately, a sort of mini-pipx that doesn't clutter your dependency tree. We also are exploring the ability to require plugins using this same mechanism so they can be installed per-project. The first iteration of that design is #5740, and it would solve the use cases outlined here without changing Poetry's fundamental design.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.