How should Lightning manage its dependencies for development?

[awaelchli]

Lightning currently maintains a requirements file of this style:

package0
package1 >= 1.0
package2 => 2.0, !=3.0  # something not good with 3.0
# package3 < 4.0  # lightning does not do that

A recurrent issue is that a package releases a new version that makes our development branch fail. Due to our strict merge policies we can't merge anything with failing tests. Until a dependency issue has been resolved, development is stale. This in my view is a security problem, as external forces can halt development at any time!

There are several approaches how to solve this problem but there are drawbacks to each of them.

Option A:
Block major versions as shown above with package3, manually update once a new release opens.
But this will prevent users from installing the new version they like.

Option B:
Automate dependency management with a bot (like https://dependabot.com/ as suggested by @carmocca @ethanwharris).
However, this will likely require us to split somehow the requirements for development vs. the ones we include in releases (not sure on this one how it works).

Option C:
Please suggest other approaches below.

We may want to look at other OS libraries how they do it :)

[Borda]

I would not block using future versions (restrict dependency versions on these present at the time of Pl is released) just when we are not sure that they are fully compatible...
This restriction may lead to deadlock when resolving feasible version intersection 🐰

However, this will likely require us to split somehow the requirements for development vs. the ones we include in releases

the edit/adjustment can be done at the release time, but still, we shall differentiate if a version is skipped because we artificially freezer dependency for development of such a package/version is really bad