Remove obsolete extension and clean up replay file extensions (frequenz-floss#118)

The `jinja2_time.TimeExtension` extension was removed in cookiecutter
2.3.0 and both this extension and the new replacement
(`cookiecutter.extensions.TimeExtension`) are pre-loaded by
cookiecutter, so it wasn't necessary in the first place. We remove it to
avoid errors when using the new cookiecutter version.
    new version.

Also we now remove the `_extensions` key from the generated replay file
for new projects as it is only likely to create more upgrade errors, and
we almost always want to just use the extensions declared by the
template itself, not our own.

Author: llucax

RELEASE_NOTES.md

@@ -10,7 +10,9 @@
 
 ### Cookiecutter template
 
-<!-- Here upgrade steps for cookiecutter specifically -->
+- If your replay file contains a `_extensions` key, you should remove it, as you most likely want to use the extensions declared by the repo-config cookiecutter template you are upgrading to, otherwise you could get errors about missing extensions.
+
+- If your replay file contains a long `Introduction` key, you can replace it with an empty string (`""`), it doesn't need to have any particular content and it increases the size and noise in the replay file.
 
 ## New Features
 
@@ -25,6 +27,20 @@
 
   This makes it easier to upgrade projects to new templates, as removing whole lines is easier than having to edit them.
 
+- Clean up `_extensions` from the generated replay file.
+
+  It is not needed in the generated project, we always want to use the ones from the repo-config template.
+
+  This should ease upgrading projects, making it less likely to have errors about missing extensions.
+
+- Clean up the `Introduction` variable from the generated replay file.
+
+  This is just a hack to be able to show a help about the template variables, keeping that text only increases the size and noise in the replay file.
+
+- Add a `\n` to the end of the replay file.
+
+  This is just to be nice to most editors and text files conventions, that likes it more if there is a `\n` at the end of the file.
+
 ## Bug Fixes
 
 <!-- Here goes notable bug fixes that are worth a special mention or explanation -->

cookiecutter/cookiecutter.json

@@ -1,37 +1,36 @@
 {
-	"Introduction": "{{cookiecutter | introduction}}",
-	"type": [
-		"actor",
-		"api",
-		"app",
-		"lib",
-		"model"
-	],
-	"name": null,
-	"description": null,
-	"title": "{{cookiecutter | title}}",
-	"keywords": "(comma separated: 'frequenz', <type> and <name> are included automatically)",
-	"github_org": "frequenz-floss",
-	"license": [
-		"MIT",
-		"Proprietary"
-	],
-	"author_name": "Frequenz Energy-as-a-Service GmbH",
-	"author_email": "[email protected]",
-	"python_package": "{{cookiecutter | python_package}}",
-	"pypi_package_name": "{{cookiecutter | pypi_package_name}}",
-	"github_repo_name": "{{cookiecutter | github_repo_name}}",
-	"default_codeowners": "(like @some-org/some-team; defaults to a team based on the repo type)",
-	"_extensions": [
-		"jinja2_time.TimeExtension",
-		"local_extensions.as_identifier",
-		"local_extensions.default_codeowners",
-		"local_extensions.github_repo_name",
-		"local_extensions.introduction",
-		"local_extensions.keywords",
-		"local_extensions.pypi_package_name",
-		"local_extensions.python_package",
-		"local_extensions.src_path",
-		"local_extensions.title"
-	]
+  "Introduction": "{{cookiecutter | introduction}}",
+  "type": [
+    "actor",
+    "api",
+    "app",
+    "lib",
+    "model"
+  ],
+  "name": null,
+  "description": null,
+  "title": "{{cookiecutter | title}}",
+  "keywords": "(comma separated: 'frequenz', <type> and <name> are included automatically)",
+  "github_org": "frequenz-floss",
+  "license": [
+    "MIT",
+    "Proprietary"
+  ],
+  "author_name": "Frequenz Energy-as-a-Service GmbH",
+  "author_email": "[email protected]",
+  "python_package": "{{cookiecutter | python_package}}",
+  "pypi_package_name": "{{cookiecutter | pypi_package_name}}",
+  "github_repo_name": "{{cookiecutter | github_repo_name}}",
+  "default_codeowners": "(like @some-org/some-team; defaults to a team based on the repo type)",
+  "_extensions": [
+    "local_extensions.as_identifier",
+    "local_extensions.default_codeowners",
+    "local_extensions.github_repo_name",
+    "local_extensions.introduction",
+    "local_extensions.keywords",
+    "local_extensions.pypi_package_name",
+    "local_extensions.python_package",
+    "local_extensions.src_path",
+    "local_extensions.title"
+  ]
 }

cookiecutter/hooks/post_gen_project.py

@@ -131,6 +131,53 @@ def finish_setup() -> None:
     commit_git_changes(first_commit=was_repo_initialized)
 
 
+def clean_replay_file(
+    *,
+    replay_data: dict[str, Any],
+    src: _pathlib.Path,  # pylint: disable=unused-argument
+    dst: _pathlib.Path,
+) -> dict[str, Any]:
+    """Clean the replay file.
+
+    Performs some cleaning of the replay file generated by cookiecutter to make it
+    more suitable for the generated project.
+
+    Args:
+        replay_data: The replay data to clean.
+        src: The path of the original replay file.
+        dst: The path of the cleaned replay file.
+
+    Returns:
+        The cleaned replay data.
+    """
+    # Remove the _output_dir key from the replay data because it is an absolute path
+    # that depends on the current environment and we don't want to add it as part of
+    # the generated project.
+    replay_data["cookiecutter"].pop("_output_dir", None)
+
+    # Remove the _extensions key from the replay data because it is not needed in the
+    # generated project, we always want to use the ones from the repo-config template.
+    # This should ease upgrading.
+    replay_data["cookiecutter"].pop("_extensions", None)
+
+    # Overwrite the Introduction (which contains just a long description of the
+    # cookiecutter variables) with an empty string to reduce the replay file size and
+    # avoid noise.
+    replay_data["cookiecutter"]["Introduction"] = ""
+
+    if template := replay_data["cookiecutter"].get("_template"):
+        if not template.startswith(("gh:", "git@", "https://")):
+            print(
+                f"WARNING: The replay file's `_template` ({template}) doesn't seem "
+                "to be a remote repository, it won't be saved to avoid storing a "
+                "local template in the new repository. If this is incorrect, "
+                f"please add it back manually to {dst}."
+            )
+            replay_data["cookiecutter"].pop("_template", None)
+
+    return replay_data
+
+
 def copy_replay_file() -> None:
     """Copy the replay file to the project root."""
     src = _pathlib.Path("~/.cookiecutter_replay/cookiecutter.json").expanduser()
@@ -146,25 +193,13 @@ def copy_replay_file() -> None:
 
     try:
         with src.open("r", encoding="utf8") as input_file:
-            replay_data = _json.load(input_file)
-
-        # Remove the _output_dir key from the replay data because it is an absolute path
-        # that depends on the current environment and we don't want to add it as part of
-        # the generated project.
-        replay_data["cookiecutter"].pop("_output_dir", None)
-
-        if template := replay_data["cookiecutter"].get("_template"):
-            if not template.startswith(("gh:", "git@", "https://")):
-                print(
-                    f"WARNING: The replay file's `_template` ({template}) doesn't seem "
-                    "to be a remote repository, it won't be saved to avoid storing a "
-                    "local template in the new repository. If this is incorrect, "
-                    f"please add it back manually to {dst}."
-                )
-                replay_data["cookiecutter"].pop("_template", None)
+            replay_data = clean_replay_file(
+                replay_data=_json.load(input_file), src=src, dst=dst
+            )
 
         with dst.open("w", encoding="utf8") as output_file:
             _json.dump(replay_data, output_file, indent=2)
+            output_file.write("\n")
     except KeyError as error:
         print(
             f"WARNING: Error parsing the replay file {src} -> {dst} ({error}). "

tests_golden/integration/test_cookiecutter_generation/actor/frequenz-actor-test/.cookiecutter-replay.json

@@ -1,6 +1,6 @@
 {
   "cookiecutter": {
-    "Introduction": "]\n\nWelcome to repo-config Cookiecutter template!\n\nThis template will help you to create a new repository for your project. You will be asked to provide some information about your project.\n\nHere is an explanation of what each variable is for and will be used for:\n\n* `type`: The type of repository. It must be chosen from the list.\n\n* `name`: The name of the project. This will be used to build defaults for\n  other inputs, such as `title`, `python_package`, etc. It should be one word,\n  using only alphanumeric characters (and starting with a letter). It can\n  include also `_` and `-` which will be handled differently when building\n  other variables from it (replaced by spaces in titles for example).\n\n* `description`: A short description of the project. It will be used as the\n  description in the `README.md`, `pyproject.toml`, `mkdocs.yml`, etc.\n\n* `title`: A human-readable name or title for the project. It will be used in\n  the `README.md`, `CONTRIBUTING.md`, and other files to refer to the project,\n  as well as the site title in `mkdocs.yml`.\n\n* `keywords`: A comma-separated list of keywords that will be used in the\n  `pyproject.toml` file. If left untouched, it will use only some predefined\n  keywords. If anything else is entered, it will be **added** to the default\n  keywords.\n\n* `github_org`: The GitHub handle of the organization where the project will\n  reside. This will be used to generate links to the project on GitHub.\n\n* `license`: Currently, only two options are provided: `MIT`, which should be\n  used for open-source projects, and `Proprietary`, which should be used for\n  closed-source projects. This will be added to file headers and used as the\n  license in `pyproject.toml`.\n\n* `author_name`, `author_email`: The name and email address of the author of\n  the project. They will be used in the copyright notice in file headers and\n  as the author in `pyproject.toml`.\n\n* `python_package`: The Python package in which this project will reside. All\n  files provided by this project should be located in this package. This needs\n  to be a list of valid Python identifiers separated by dots. The source file\n  structure will be derived from this. For example, `frequenz.actor.example`\n  will generate files in `src/frequenz/actor/example`.\n\n* `pypi_package_name`: The name of the PyPI/wheel/distribution package. This\n  should be consistent with the `python_package`, usually replacing `.` with\n  `-`. For example, `frequenz-actor-example`.\n\n* `github_repo_name`: The handle of the GitHub repository where the project\n  will reside. This will be used to generate links to the project on GitHub and\n  as the top-level directory name.\n\n* `default_codeowners`: A space-separated list of GitHub teams (`@org/team`) or\n  users (`@user`) that will be the default code owners for this project. This\n  will be used to build the `CODEOWNERS` file. Please refer to the [code owners\n  documentation] for more details on the valid syntax.\n\n[code owners documentation]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners\n\n\n[Please press any key to continue",
+    "Introduction": "",
     "type": "actor",
     "name": "test",
     "description": "Test description",
@@ -13,18 +13,6 @@
     "python_package": "frequenz.actor.test",
     "pypi_package_name": "frequenz-actor-test",
     "github_repo_name": "frequenz-actor-test",
-    "default_codeowners": "(like @some-org/some-team; defaults to a team based on the repo type)",
-    "_extensions": [
-      "jinja2_time.TimeExtension",
-      "local_extensions.as_identifier",
-      "local_extensions.default_codeowners",
-      "local_extensions.github_repo_name",
-      "local_extensions.introduction",
-      "local_extensions.keywords",
-      "local_extensions.pypi_package_name",
-      "local_extensions.python_package",
-      "local_extensions.src_path",
-      "local_extensions.title"
-    ]
+    "default_codeowners": "(like @some-org/some-team; defaults to a team based on the repo type)"
   }
-}
+}

tests_golden/integration/test_cookiecutter_generation/api/frequenz-api-test/.cookiecutter-replay.json

@@ -1,6 +1,6 @@
 {
   "cookiecutter": {
-    "Introduction": "]\n\nWelcome to repo-config Cookiecutter template!\n\nThis template will help you to create a new repository for your project. You will be asked to provide some information about your project.\n\nHere is an explanation of what each variable is for and will be used for:\n\n* `type`: The type of repository. It must be chosen from the list.\n\n* `name`: The name of the project. This will be used to build defaults for\n  other inputs, such as `title`, `python_package`, etc. It should be one word,\n  using only alphanumeric characters (and starting with a letter). It can\n  include also `_` and `-` which will be handled differently when building\n  other variables from it (replaced by spaces in titles for example).\n\n* `description`: A short description of the project. It will be used as the\n  description in the `README.md`, `pyproject.toml`, `mkdocs.yml`, etc.\n\n* `title`: A human-readable name or title for the project. It will be used in\n  the `README.md`, `CONTRIBUTING.md`, and other files to refer to the project,\n  as well as the site title in `mkdocs.yml`.\n\n* `keywords`: A comma-separated list of keywords that will be used in the\n  `pyproject.toml` file. If left untouched, it will use only some predefined\n  keywords. If anything else is entered, it will be **added** to the default\n  keywords.\n\n* `github_org`: The GitHub handle of the organization where the project will\n  reside. This will be used to generate links to the project on GitHub.\n\n* `license`: Currently, only two options are provided: `MIT`, which should be\n  used for open-source projects, and `Proprietary`, which should be used for\n  closed-source projects. This will be added to file headers and used as the\n  license in `pyproject.toml`.\n\n* `author_name`, `author_email`: The name and email address of the author of\n  the project. They will be used in the copyright notice in file headers and\n  as the author in `pyproject.toml`.\n\n* `python_package`: The Python package in which this project will reside. All\n  files provided by this project should be located in this package. This needs\n  to be a list of valid Python identifiers separated by dots. The source file\n  structure will be derived from this. For example, `frequenz.actor.example`\n  will generate files in `src/frequenz/actor/example`.\n\n* `pypi_package_name`: The name of the PyPI/wheel/distribution package. This\n  should be consistent with the `python_package`, usually replacing `.` with\n  `-`. For example, `frequenz-actor-example`.\n\n* `github_repo_name`: The handle of the GitHub repository where the project\n  will reside. This will be used to generate links to the project on GitHub and\n  as the top-level directory name.\n\n* `default_codeowners`: A space-separated list of GitHub teams (`@org/team`) or\n  users (`@user`) that will be the default code owners for this project. This\n  will be used to build the `CODEOWNERS` file. Please refer to the [code owners\n  documentation] for more details on the valid syntax.\n\n[code owners documentation]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners\n\n\n[Please press any key to continue",
+    "Introduction": "",
     "type": "api",
     "name": "test",
     "description": "Test description",
@@ -13,18 +13,6 @@
     "python_package": "frequenz.api.test",
     "pypi_package_name": "frequenz-api-test",
     "github_repo_name": "frequenz-api-test",
-    "default_codeowners": "(like @some-org/some-team; defaults to a team based on the repo type)",
-    "_extensions": [
-      "jinja2_time.TimeExtension",
-      "local_extensions.as_identifier",
-      "local_extensions.default_codeowners",
-      "local_extensions.github_repo_name",
-      "local_extensions.introduction",
-      "local_extensions.keywords",
-      "local_extensions.pypi_package_name",
-      "local_extensions.python_package",
-      "local_extensions.src_path",
-      "local_extensions.title"
-    ]
+    "default_codeowners": "(like @some-org/some-team; defaults to a team based on the repo type)"
   }
-}
+}

tests_golden/integration/test_cookiecutter_generation/app/frequenz-app-test/.cookiecutter-replay.json

@@ -1,6 +1,6 @@
 {
   "cookiecutter": {
-    "Introduction": "]\n\nWelcome to repo-config Cookiecutter template!\n\nThis template will help you to create a new repository for your project. You will be asked to provide some information about your project.\n\nHere is an explanation of what each variable is for and will be used for:\n\n* `type`: The type of repository. It must be chosen from the list.\n\n* `name`: The name of the project. This will be used to build defaults for\n  other inputs, such as `title`, `python_package`, etc. It should be one word,\n  using only alphanumeric characters (and starting with a letter). It can\n  include also `_` and `-` which will be handled differently when building\n  other variables from it (replaced by spaces in titles for example).\n\n* `description`: A short description of the project. It will be used as the\n  description in the `README.md`, `pyproject.toml`, `mkdocs.yml`, etc.\n\n* `title`: A human-readable name or title for the project. It will be used in\n  the `README.md`, `CONTRIBUTING.md`, and other files to refer to the project,\n  as well as the site title in `mkdocs.yml`.\n\n* `keywords`: A comma-separated list of keywords that will be used in the\n  `pyproject.toml` file. If left untouched, it will use only some predefined\n  keywords. If anything else is entered, it will be **added** to the default\n  keywords.\n\n* `github_org`: The GitHub handle of the organization where the project will\n  reside. This will be used to generate links to the project on GitHub.\n\n* `license`: Currently, only two options are provided: `MIT`, which should be\n  used for open-source projects, and `Proprietary`, which should be used for\n  closed-source projects. This will be added to file headers and used as the\n  license in `pyproject.toml`.\n\n* `author_name`, `author_email`: The name and email address of the author of\n  the project. They will be used in the copyright notice in file headers and\n  as the author in `pyproject.toml`.\n\n* `python_package`: The Python package in which this project will reside. All\n  files provided by this project should be located in this package. This needs\n  to be a list of valid Python identifiers separated by dots. The source file\n  structure will be derived from this. For example, `frequenz.actor.example`\n  will generate files in `src/frequenz/actor/example`.\n\n* `pypi_package_name`: The name of the PyPI/wheel/distribution package. This\n  should be consistent with the `python_package`, usually replacing `.` with\n  `-`. For example, `frequenz-actor-example`.\n\n* `github_repo_name`: The handle of the GitHub repository where the project\n  will reside. This will be used to generate links to the project on GitHub and\n  as the top-level directory name.\n\n* `default_codeowners`: A space-separated list of GitHub teams (`@org/team`) or\n  users (`@user`) that will be the default code owners for this project. This\n  will be used to build the `CODEOWNERS` file. Please refer to the [code owners\n  documentation] for more details on the valid syntax.\n\n[code owners documentation]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners\n\n\n[Please press any key to continue",
+    "Introduction": "",
     "type": "app",
     "name": "test",
     "description": "Test description",
@@ -13,18 +13,6 @@
     "python_package": "frequenz.app.test",
     "pypi_package_name": "frequenz-app-test",
     "github_repo_name": "frequenz-app-test",
-    "default_codeowners": "(like @some-org/some-team; defaults to a team based on the repo type)",
-    "_extensions": [
-      "jinja2_time.TimeExtension",
-      "local_extensions.as_identifier",
-      "local_extensions.default_codeowners",
-      "local_extensions.github_repo_name",
-      "local_extensions.introduction",
-      "local_extensions.keywords",
-      "local_extensions.pypi_package_name",
-      "local_extensions.python_package",
-      "local_extensions.src_path",
-      "local_extensions.title"
-    ]
+    "default_codeowners": "(like @some-org/some-team; defaults to a team based on the repo type)"
   }
-}
+}