Revising Dependabot config and version specifiers of dev dependencies

[sisp]

I've stumbled over Dependabot's current behavior in this project and wonder whether it should be revised.

As far as I can tell (it's difficult to verify), Dependabot's behavior given our quite minimal configuration

copier/.github/dependabot.yml

Lines 8 to 11 in a5da81c

	- package-ecosystem: "pip"
	directory: "/"
	schedule:
	interval: "daily"

(with regard to the relevant settings) is this:

• allow defaults to direct. This means, all (runtime and dev) dependencies listed in pyproject.toml are checked for new versions, but indirect/transitive dependencies are ignored.
• versioning-strategy defaults to lockfile-only. This means, only poetry.lock is updated and pyproject.toml is never updated.

The combination of those two settings means, all (runtime and dev) dependencies listed in pyproject.toml are checked and their versions are updated only in poetry.lock when new versions are available. It also means, none of the indirect/transitive dependencies are updated (unless Dependabot detects security vulnerabilities), so while the direct dependencies remain up-to-date, (typically) the majority of dependencies in the dependency tree remains outdated. I fail to see the rationale for such behavior.

Also, I wonder about the motivation for using a version specifier other than == (or equivalently, none), i.e. exact pinning, for all dev dependencies because dev dependencies are always installed from the lock file, i.e. pinned versions, so specifying lower bounds has little meaning except it increases the search space of the dependency resolver when the minimum version isn't the most recent version. And for code quality tools like formatters or linters, which may introduce, e.g., new rules even in non-major releases resulting in different formatting/linting output, pinning in combination with explicit version updates (assisted by Dependabot), which require/incur changes in the code to satisfy, e.g., new rules, seems much more sensible to me than using lower bounds and having the poetry.lock versions diverge from the minimum versions in pyproject.toml.

To summarize, I wonder whether the following changes would be sensible:

• Use pinned versions for dev dependencies in pyproject.toml.

• Update .github/dependabot.yml

  • to update all (i.e. also indirect/transitive) dependencies in poetry.lock, and
  • to update pinned dev dependencies in pyproject.toml

  which might look like this (untested):

   - package-ecosystem: "pip"
     directory: "/"
     schedule:
       interval: "daily"
  +  allow:
  +    - dependency-type: all
  +  versioning-strategy: lockfile-only
  +
  +- package-ecosystem: "pip"
  +  directory: "/"
  +  schedule:
  +    interval: "daily"
  +  allow:
  +    - dependency-type: development
  +  versioning-strategy: increase

  Here, I assume that versioning-strategy: increase increases also pinned versions and also to new major versions. If pinning in pyproject.toml blocks versioning-strategy: increase from doing anything, then we could resort to using lower bounds.

[sisp]

Odd, specifying package-ecosystem: "pip" twice in this way seems to be disallowed, giving the following error:

The property '#/updates/2' is a duplicate. Update configs must have a unique combination of 'package-ecosystem', 'directory', and 'target-branch'

So it seems to be impossible to have different versioning strategies for dependency types. 🤔 Why should this not be allowed? 🤷

[sisp]

Seems related to dependabot/dependabot-core#1778. I'm shocked that Dependabot does not support this. 😱 Perhaps Renovate Bot is worth looking into? It supports lock file maintenance:

You can use lockFileMaintenance to refresh lock files to keep them up-to-date.

When Renovate performs lockFileMaintenance it deletes the lock file and runs the relevant package manager. That package manager creates a new lock file, where all dependency versions are updated to the latest version. Renovate then commits that lock file to the update branch and creates the lock file update PR.

[yajo]

I agree to update all, and not only direct dependencies. However that will mean having more Dependabot PRs. Not really a big problem, but it would be nice to automerge if green somehow.

I'd like to keep the current strategy. It seems correct to me to update just the lock file if possible, but to update the manifest if needed. That's happening in #1243 for example.

Lower bounds in dev dependencies are not really used, but also not harmful. We could leave them as they are, or just use *.

[sisp]

I believe I could live with using * as the version specifier for dev dependencies. The only downside, IMO, is the reduced transparency which version of a dev dependency is actually being used when not specifying a version pin. But Dependabot seems to be incapable of updating the pins in the manifest only for dev dependencies and updating the lock file for all (i.e. also indirect) dependencies.

So the change your suggesting is this, right?

• Update the version specifiers of all dev dependencies to *.

• Update the Dependabot config:

   - package-ecosystem: "pip"
     directory: "/"
     schedule:
       interval: "daily"
  +  allow:
  +    - dependency-type: all

  So we'd keep the versioning strategy but extend the lock file updates also to indirect dependencies, if I read the Dependabot docs right.

Regarding the amount of Dependabot PRs, I think it might be nice to keep individual PRs for updates of direct dependencies but have a single PR for a all indirect dependencies. But I don't think Dependabot supports this either. Even if we auto-merged Dependabot PRs upon passing CI checks, those commits would pollute the Git history quite a bit. 🤔

[yajo]

Having atomic updates is better IMHO. For example, sometimes I need to tweak something for the nix build, and it's easier that way.

BTW what's the downside of having lower bounds in dev deps? 🤔

[sisp]

BTW what's the downside of having lower bounds in dev deps? 🤔

I think the result is typically fine because the versions are locked and typically only increased over time, but the lower bound is often arbitrary (e.g. the latest version at the time when the dependency got added) and may become wrong (e.g. when using a more recent feature without raising the lower bound to the version in which it got introduced, or e.g. when using a formatter that formats code differently in a recent version than in an older version and you wouldn't notice because you're on the newer version now). So, my point is rather about theoretical than practical correctness, yet it bothers me. 😆

[yajo]

In general terms I'm not much in favor of changing things just because yes. If something isn't broken, I prefer to not fix it.

Anyways I don't wanna be too dogmatic about it, so if you prefer we can put those asterisks in the dependencies.

The other improvement actually adds value so it's more important IMHO.

[sisp]

@yajo Are you aware that Renovate Bot supports updating Nix dependencies? Dependabot doesn't. Just thinking whether checking out Renovate Bot might be worth some time, also regarding the original topic, but also because of Nix support.

[yajo]

It'd love that!

I actually asked for support on dependabot some time ago: dependabot/dependabot-core#7340, but it seems stale.