Exclude frequenz-repo-config from dependabot grouping (#368)

Since we are still in development (at branch v0.x.x), breaking changes
can be introduced by minor (and even patch) updates, blocking many
dependabot PRs as updates of this library fail or need manual
intervention.

With this `frequenz-repo-config` updates will be done in a separate PR.

Fixes #366.

Author: llucax

.github/dependabot.yml

@@ -13,23 +13,36 @@ updates:
     versioning-strategy: auto
     # Allow up to 10 open pull requests for updates to dependency versions
     open-pull-requests-limit: 10
-    # Group production and development (required and optional in the context of
-    # pyproject.toml) dependency updates when they are patch and minor updates,
-    # so we end up with less PRs being generated.
-    # Major updates are still managed, but they'll create one PR per
-    # dependency, as major updates are expected to be breaking, it is better to
-    # manage them individually.
+    # We group patch updates as they should always work.
+    # We also group minor updates, as it works too for most libraries,
+    # typically except libraries that don't have a stable release yet (v0.x.x
+    # branch), so we make some exceptions for them.
+    # Major updates and dependencies excluded by the above groups are still
+    # managed, but they'll create one PR per dependency, as breakage is
+    # expected, so it might need manual intervention.
+    # Finally, we group some dependencies that are related to each other, and
+    # usually need to be updated together.
     groups:
-      required:
-        dependency-type: "production"
+      patch:
         update-types:
-          - "minor"
           - "patch"
-      optional:
-        dependency-type: "development"
+        exclude-patterns:
+          # pydoclint has shipped breaking changes in patch updates often
+          - "pydoclint"
+      minor:
         update-types:
           - "minor"
-          - "patch"
+        exclude-patterns:
+          - "async-solipsism"
+          - "markdown-callouts"
+          - "mkdocs-gen-files"
+          - "mkdocs-literate-nav"
+          - "mkdocstrings*"
+          - "pydoclint"
+          - "pytest-asyncio"
+      mkdocstrings:
+        patterns:
+          - "mkdocstrings*"
 
   - package-ecosystem: "github-actions"
     directory: "/"

RELEASE_NOTES.md

@@ -26,7 +26,12 @@ But you might still need to adapt your code:
 
 ### Cookiecutter template
 
-<!-- Here new features for cookiecutter specifically -->
+- Dependabot config now uses a new grouping that should make upgrades more smooth.
+
+    * We group patch updates as they should always work.
+    * We also group minor updates, as it works too for most libraries, typically except libraries that don't have a stable release yet (v0.x.x branch), so we make some exceptions for them.
+    * Major updates and dependencies excluded by the above groups are still managed, but they'll create one PR per dependency, as breakage is expected, so it might need manual intervention.
+    * Finally, we group some dependencies that are related to each other, and usually needs to be updated together.
 
 ## Bug Fixes
 

cookiecutter/migrate.py

@@ -32,10 +32,107 @@ def main() -> None:
     """Run the migration steps."""
     # Add a separation line like this one after each migration step.
     print("=" * 72)
+    regroup_dependabot()
+    print("=" * 72)
     print("Migration script finished. Remember to follow any manual instructions.")
     print("=" * 72)
 
 
+def regroup_dependabot() -> None:
+    """Use new dependabot groups to separate dependencies that break often."""
+    print("Using new dependabot groups to separate dependencies that break often...")
+    # Dependabot configuration file
+    dependabot_file = Path(".github/dependabot.yml")
+
+    # Skip if the file doesn't exist
+    if not dependabot_file.exists():
+        manual_step(
+            "Dependabot configuration file not found, not excluding "
+            "frequenz-repo-config from group updates. Please consider adding a "
+            "dependabot configuration file."
+        )
+        return
+
+    dependabot_content = dependabot_file.read_text(encoding="utf-8")
+
+    new_groups = """\
+    # We group patch updates as they should always work.
+    # We also group minor updates, as it works too for most libraries,
+    # typically except libraries that don't have a stable release yet (v0.x.x
+    # branch), so we make some exceptions for them.
+    # Major updates and dependencies excluded by the above groups are still
+    # managed, but they'll create one PR per dependency, as breakage is
+    # expected, so it might need manual intervention.
+    # Finally, we group some dependencies that are related to each other, and
+    # usually need to be updated together.
+    groups:
+      patch:
+        update-types:
+          - "patch"
+        exclude-patterns:
+          # pydoclint has shipped breaking changes in patch updates often
+          - "pydoclint"
+      minor:
+        update-types:
+          - "minor"
+        exclude-patterns:
+          - "async-solipsism"
+          - "frequenz-repo-config*"
+          - "markdown-callouts"
+          - "mkdocs-gen-files"
+          - "mkdocs-literate-nav"
+          - "mkdocstrings*"
+          - "pydoclint"
+          - "pytest-asyncio"
+      # We group repo-config updates as it uses optional dependencies that are
+      # considered different dependencies otherwise, and will create one PR for
+      # each if we don't group them.
+      repo-config:
+        patterns:
+          - "frequenz-repo-config*"
+      mkdocstrings:
+        patterns:
+          - "mkdocstrings*"
+"""
+
+    marker = "    open-pull-requests-limit: 10"
+    if marker not in dependabot_content:
+        manual_step(
+            f"Could not file marker ({marker!r}) in {dependabot_file}, "
+            "can't update automatically. Please consider using these new groups "
+            "in the dependabot configuration file:"
+        )
+        return
+
+    text_to_replace = ""
+    found_marker = False
+    for line in dependabot_content.splitlines():
+        if line == marker:
+            found_marker = True
+            continue
+        if not found_marker:
+            continue
+        if line == "" and found_marker:
+            break
+        text_to_replace += line + "\n"
+
+    if not text_to_replace:
+        manual_step(
+            "Could not find the text to replace with the new depenndabot "
+            "groups. Please consider using these new groups in the dependabot "
+            "configuration file:"
+        )
+        return
+
+    replace_file_contents_atomically(
+        dependabot_file,
+        text_to_replace,
+        new_groups,
+        count=1,
+        content=dependabot_content,
+    )
+
+
 def apply_patch(patch_content: str) -> None:
     """Apply a patch using the patch utility."""
     subprocess.run(["patch", "-p1"], input=patch_content.encode(), check=True)

cookiecutter/{{cookiecutter.github_repo_name}}/.github/dependabot.yml

@@ -13,23 +13,46 @@ updates:
     versioning-strategy: auto
     # Allow up to 10 open pull requests for updates to dependency versions
     open-pull-requests-limit: 10
-    # We group production and development ("optional" in the context of
-    # pyproject.toml) dependency updates when they are patch and minor updates,
-    # so we end up with less PRs being generated.
-    # Major updates are still managed, but they'll create one PR per
-    # dependency, as major updates are expected to be breaking, it is better to
-    # manage them individually.
+    # We group patch updates as they should always work.
+    # We also group minor updates, as it works too for most libraries,
+    # typically except libraries that don't have a stable release yet (v0.x.x
+    # branch), so we make some exceptions for them.
+    # Major updates and dependencies excluded by the above groups are still
+    # managed, but they'll create one PR per dependency, as breakage is
+    # expected, so it might need manual intervention.
+    # Finally, we group some dependencies that are related to each other, and
+    # usually need to be updated together.
     groups:
-      required:
-        dependency-type: "production"
+      patch:
         update-types:
-          - "minor"
           - "patch"
-      optional:
-        dependency-type: "development"
+        exclude-patterns:
+          # pydoclint has shipped breaking changes in patch updates often
+          - "pydoclint"
+      minor:
         update-types:
           - "minor"
-          - "patch"
+        exclude-patterns:
+          - "async-solipsism"
+{%- if cookiecutter.type == "api" %}
+          - "frequenz-api-common"
+{%- endif %}
+          - "frequenz-repo-config*"
+          - "markdown-callouts"
+          - "mkdocs-gen-files"
+          - "mkdocs-literate-nav"
+          - "mkdocstrings*"
+          - "pydoclint"
+          - "pytest-asyncio"
+      # We group repo-config updates as it uses optional dependencies that are
+      # considered different dependencies otherwise, and will create one PR for
+      # each if we don't group them.
+      repo-config:
+        patterns:
+          - "frequenz-repo-config*"
+      mkdocstrings:
+        patterns:
+          - "mkdocstrings*"
 
   - package-ecosystem: "github-actions"
     directory: "/"

tests_golden/integration/test_cookiecutter_generation/actor/frequenz-actor-test/.github/dependabot.yml

@@ -13,23 +13,43 @@ updates:
     versioning-strategy: auto
     # Allow up to 10 open pull requests for updates to dependency versions
     open-pull-requests-limit: 10
-    # We group production and development ("optional" in the context of
-    # pyproject.toml) dependency updates when they are patch and minor updates,
-    # so we end up with less PRs being generated.
-    # Major updates are still managed, but they'll create one PR per
-    # dependency, as major updates are expected to be breaking, it is better to
-    # manage them individually.
+    # We group patch updates as they should always work.
+    # We also group minor updates, as it works too for most libraries,
+    # typically except libraries that don't have a stable release yet (v0.x.x
+    # branch), so we make some exceptions for them.
+    # Major updates and dependencies excluded by the above groups are still
+    # managed, but they'll create one PR per dependency, as breakage is
+    # expected, so it might need manual intervention.
+    # Finally, we group some dependencies that are related to each other, and
+    # usually need to be updated together.
     groups:
-      required:
-        dependency-type: "production"
+      patch:
         update-types:
-          - "minor"
           - "patch"
-      optional:
-        dependency-type: "development"
+        exclude-patterns:
+          # pydoclint has shipped breaking changes in patch updates often
+          - "pydoclint"
+      minor:
         update-types:
           - "minor"
-          - "patch"
+        exclude-patterns:
+          - "async-solipsism"
+          - "frequenz-repo-config*"
+          - "markdown-callouts"
+          - "mkdocs-gen-files"
+          - "mkdocs-literate-nav"
+          - "mkdocstrings*"
+          - "pydoclint"
+          - "pytest-asyncio"
+      # We group repo-config updates as it uses optional dependencies that are
+      # considered different dependencies otherwise, and will create one PR for
+      # each if we don't group them.
+      repo-config:
+        patterns:
+          - "frequenz-repo-config*"
+      mkdocstrings:
+        patterns:
+          - "mkdocstrings*"
 
   - package-ecosystem: "github-actions"
     directory: "/"

tests_golden/integration/test_cookiecutter_generation/api/frequenz-api-test/.github/dependabot.yml

@@ -13,23 +13,44 @@ updates:
     versioning-strategy: auto
     # Allow up to 10 open pull requests for updates to dependency versions
     open-pull-requests-limit: 10
-    # We group production and development ("optional" in the context of
-    # pyproject.toml) dependency updates when they are patch and minor updates,
-    # so we end up with less PRs being generated.
-    # Major updates are still managed, but they'll create one PR per
-    # dependency, as major updates are expected to be breaking, it is better to
-    # manage them individually.
+    # We group patch updates as they should always work.
+    # We also group minor updates, as it works too for most libraries,
+    # typically except libraries that don't have a stable release yet (v0.x.x
+    # branch), so we make some exceptions for them.
+    # Major updates and dependencies excluded by the above groups are still
+    # managed, but they'll create one PR per dependency, as breakage is
+    # expected, so it might need manual intervention.
+    # Finally, we group some dependencies that are related to each other, and
+    # usually need to be updated together.
     groups:
-      required:
-        dependency-type: "production"
+      patch:
         update-types:
-          - "minor"
           - "patch"
-      optional:
-        dependency-type: "development"
+        exclude-patterns:
+          # pydoclint has shipped breaking changes in patch updates often
+          - "pydoclint"
+      minor:
         update-types:
           - "minor"
-          - "patch"
+        exclude-patterns:
+          - "async-solipsism"
+          - "frequenz-api-common"
+          - "frequenz-repo-config*"
+          - "markdown-callouts"
+          - "mkdocs-gen-files"
+          - "mkdocs-literate-nav"
+          - "mkdocstrings*"
+          - "pydoclint"
+          - "pytest-asyncio"
+      # We group repo-config updates as it uses optional dependencies that are
+      # considered different dependencies otherwise, and will create one PR for
+      # each if we don't group them.
+      repo-config:
+        patterns:
+          - "frequenz-repo-config*"
+      mkdocstrings:
+        patterns:
+          - "mkdocstrings*"
 
   - package-ecosystem: "github-actions"
     directory: "/"

tests_golden/integration/test_cookiecutter_generation/app/frequenz-app-test/.github/dependabot.yml

@@ -13,23 +13,43 @@ updates:
     versioning-strategy: auto
     # Allow up to 10 open pull requests for updates to dependency versions
     open-pull-requests-limit: 10
-    # We group production and development ("optional" in the context of
-    # pyproject.toml) dependency updates when they are patch and minor updates,
-    # so we end up with less PRs being generated.
-    # Major updates are still managed, but they'll create one PR per
-    # dependency, as major updates are expected to be breaking, it is better to
-    # manage them individually.
+    # We group patch updates as they should always work.
+    # We also group minor updates, as it works too for most libraries,
+    # typically except libraries that don't have a stable release yet (v0.x.x
+    # branch), so we make some exceptions for them.
+    # Major updates and dependencies excluded by the above groups are still
+    # managed, but they'll create one PR per dependency, as breakage is
+    # expected, so it might need manual intervention.
+    # Finally, we group some dependencies that are related to each other, and
+    # usually need to be updated together.
     groups:
-      required:
-        dependency-type: "production"
+      patch:
         update-types:
-          - "minor"
           - "patch"
-      optional:
-        dependency-type: "development"
+        exclude-patterns:
+          # pydoclint has shipped breaking changes in patch updates often
+          - "pydoclint"
+      minor:
         update-types:
           - "minor"
-          - "patch"
+        exclude-patterns:
+          - "async-solipsism"
+          - "frequenz-repo-config*"
+          - "markdown-callouts"
+          - "mkdocs-gen-files"
+          - "mkdocs-literate-nav"
+          - "mkdocstrings*"
+          - "pydoclint"
+          - "pytest-asyncio"
+      # We group repo-config updates as it uses optional dependencies that are
+      # considered different dependencies otherwise, and will create one PR for
+      # each if we don't group them.
+      repo-config:
+        patterns:
+          - "frequenz-repo-config*"
+      mkdocstrings:
+        patterns:
+          - "mkdocstrings*"
 
   - package-ecosystem: "github-actions"
     directory: "/"