chore(refactoring): Introduce a Python CLI app layout (#738)

* Introduce a Python CLI app layout

This includes a structure with purpose-based modules and a standard
mechanism for adding more subcommands.

When adding a new subcommand, one has to wire the `invoke_cli_app()`
and `populate_argument_parser()` from their new module into the
mappings defined in `_cli_subcommands.py` and `_cli_parsing.py`
respectively. This is the only integration point necessary.

`populate_argument_parser()` accepts a subparser instance of
`argparse.ArgumentParser()` that a new subcommand would need to attach
new arguments into. It does not need to return anything. And the
`invoke_cli_app()` hook is called with an instance of
`argparse.Namespace()` with all the arguments parsed and pre-processed.
This function is supposed to have the main check logic and return an
instance of `._structs.ReturnCode()` or `int`.

* Add a maintainer's manual into the importable package dir

* Simplify subcommand integration to one place

Previously, adding a new subcommand required importing it into two
separate Python modules, adding them into two mappings, maintaining
the same dictionary key string. This is prone to human error and is
underintegrated.

This patch makes it easier by defining a required subcommand module
shape on the typing level. Now, one just need to implement two hook-
functions and a constant with specific signatures and import it in
a single place.

* Drop the `__main__` check per Python docs

See https://docs.python.org/3/library/__main__.html#id1.

Author: webknjaz

.pre-commit-hooks.yaml

@@ -37,7 +37,7 @@
   name: Terraform docs (overwrite README.md)
   description: Overwrite content of README.md with terraform-docs.
   require_serial: true
-  entry: terraform_docs_replace
+  entry: python -Im pre_commit_terraform replace-docs
   language: python
   files: (\.tf)$
   exclude: \.terraform/.*$

pyproject.toml

@@ -41,6 +41,3 @@ email = '[email protected]'
 [project.readme]
 file = 'README.md'
 content-type = 'text/markdown'
-
-[project.scripts]
-terraform_docs_replace = 'pre_commit_terraform.terraform_docs_replace:main'

src/pre_commit_terraform/README.md

@@ -0,0 +1,93 @@
+# Maintainer's manual
+
+## Structure
+
+This folder is what's called an [importable package]. It's a top-level folder
+that ends up being installed into `site-packages/` of virtualenvs.
+
+When the Git repository is `pip install`ed, this [import package] becomes
+available for use within respective Python interpreter instance. It can be
+imported and sub-modules can be imported through the dot-syntax. Additionally,
+the modules within can import the neighboring ones using relative imports that
+have a leading dot in them.
+
+It additionally implements a [runpy interface], meaning that its name can
+be passed to `python -m` to invoke the CLI. This is the primary method of
+integration with the [`pre-commit` framework] and local development/testing.
+
+The layout allows for having several Python modules wrapping third-party tools,
+each having an argument parser and being a subcommand for the main CLI
+interface.
+
+## Control flow
+
+When `python -m pre_commit_terraform` is executed, it imports `__main__.py`.
+Which in turn, performs the initialization of the main argument parser and the
+parsers of subcommands, followed by executing the logic defined in dedicated
+subcommand modules.
+
+## Integrating a new subcommand
+
+1. Create a new module called `subcommand_x.py`.
+2. Within that module, define two functions —
+   `invoke_cli_app(parsed_cli_args: Namespace) -> ReturnCodeType | int` and
+   `populate_argument_parser(subcommand_parser: ArgumentParser) -> None`.
+   Additionally, define a module-level constant
+   `CLI_SUBCOMMAND_NAME: Final[str] = 'subcommand-x'`.
+3. Edit [`_cli_subcommands.py`], importing `subcommand_x` as a relative module
+   and add it into the `SUBCOMMAND_MODULES` list.
+4. Edit [`.pre-commit-hooks.yaml`], adding a new hook that invokes
+   `python -m pre_commit_terraform subcommand-x`.
+
+## Manual testing
+
+Usually, having a development virtualenv where you `pip install -e .` is enough
+to make it possible to invoke the CLI app. Do so first. Most source code
+updates do not require running it again. But sometimes, it's needed.
+
+Once done, you can run `python -m pre_commit_terraform` and/or
+`python -m pre_commit_terraform subcommand-x` to see how it behaves. There's
+`--help` and all other typical conventions one would usually expect from a
+POSIX-inspired CLI app.
+
+## DX/UX considerations
+
+Since it's an app that can be executed outside the [`pre-commit` framework],
+it is useful to check out and follow these [CLI guidelines][clig].
+
+## Subcommand development
+
+`populate_argument_parser()` accepts a regular instance of
+[`argparse.ArgumentParser`]. Call its methods to extend the CLI arguments that
+would be specific for the subcommand you are creating. Those arguments will be
+available later, as an argument to the `invoke_cli_app()` function — through an
+instance of [`argparse.Namespace`]. For the `CLI_SUBCOMMAND_NAME` constant,
+choose `kebab-space-sub-command-style`, it does not need to be `snake_case`.
+
+Make sure to return a `ReturnCode` instance or an integer from
+`invoke_cli_app()`. Returning a non-zero value will result in the CLI app
+exiting with a return code typically interpreted as an error while zero means
+success. You can `import errno` to use typical POSIX error codes through their
+human-readable identifiers.
+
+Another way to interrupt the CLI app control flow is by raising an instance of
+one of the in-app errors. `raise PreCommitTerraformExit` for a successful exit,
+but it can be turned into an error outcome via
+`raise PreCommitTerraformExit(1)`.
+`raise PreCommitTerraformRuntimeError('The world is broken')` to indicate
+problems within the runtime. The framework will intercept any exceptions
+inheriting `PreCommitTerraformBaseError`, so they won't be presented to the
+end-users.
+
+[`.pre-commit-hooks.yaml`]: ../../.pre-commit-hooks.yaml
+[`_cli_parsing.py`]: ./_cli_parsing.py
+[`_cli_subcommands.py`]: ./_cli_subcommands.py
+[`argparse.ArgumentParser`]:
+https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser
+[`argparse.Namespace`]:
+https://docs.python.org/3/library/argparse.html#argparse.Namespace
+[clig]: https://clig.dev
+[importable package]: https://docs.python.org/3/tutorial/modules.html#packages
+[import package]: https://packaging.python.org/en/latest/glossary/#term-Import-Package
+[`pre-commit` framework]: https://pre-commit.com
+[runpy interface]: https://docs.python.org/3/library/__main__.html

src/pre_commit_terraform/__main__.py

@@ -0,0 +1,9 @@
+"""A runpy-style CLI entry-point module."""
+
+from sys import argv, exit as exit_with_return_code
+
+from ._cli import invoke_cli_app
+
+
+return_code = invoke_cli_app(argv[1:])
+exit_with_return_code(return_code)

src/pre_commit_terraform/_cli.py

@@ -0,0 +1,50 @@
+"""Outer CLI layer of the app interface."""
+
+from sys import stderr
+
+from ._cli_parsing import initialize_argument_parser
+from ._errors import (
+    PreCommitTerraformBaseError,
+    PreCommitTerraformExit,
+    PreCommitTerraformRuntimeError,
+)
+from ._structs import ReturnCode
+from ._types import ReturnCodeType
+
+
+def invoke_cli_app(cli_args: list[str]) -> ReturnCodeType:
+    """Run the entry-point of the CLI app.
+
+    Includes initializing parsers of all the sub-apps and
+    choosing what to execute.
+    """
+    root_cli_parser = initialize_argument_parser()
+    parsed_cli_args = root_cli_parser.parse_args(cli_args)
+
+    try:
+        return parsed_cli_args.invoke_cli_app(parsed_cli_args)
+    except PreCommitTerraformExit as exit_err:
+        print(f'App exiting: {exit_err !s}', file=stderr)
+        raise
+    except PreCommitTerraformRuntimeError as unhandled_exc:
+        print(
+            f'App execution took an unexpected turn: {unhandled_exc !s}. '
+            'Exiting...',
+            file=stderr,
+        )
+        return ReturnCode.ERROR
+    except PreCommitTerraformBaseError as unhandled_exc:
+        print(
+            f'A surprising exception happened: {unhandled_exc !s}. Exiting...',
+            file=stderr,
+        )
+        return ReturnCode.ERROR
+    except KeyboardInterrupt as ctrl_c_exc:
+        print(
+            f'User-initiated interrupt: {ctrl_c_exc !s}. Exiting...',
+            file=stderr,
+        )
+        return ReturnCode.ERROR
+
+
+__all__ = ('invoke_cli_app',)

src/pre_commit_terraform/_cli_parsing.py

@@ -0,0 +1,39 @@
+"""Argument parser initialization logic.
+
+This defines helpers for setting up both the root parser and the parsers
+of all the sub-commands.
+"""
+
+from argparse import ArgumentParser
+
+from ._cli_subcommands import SUBCOMMAND_MODULES
+
+
+def attach_subcommand_parsers_to(root_cli_parser: ArgumentParser, /) -> None:
+    """Connect all sub-command parsers to the given one.
+
+    This functions iterates over a mapping of subcommands to their
+    respective population functions, executing them to augment the
+    main parser.
+    """
+    subcommand_parsers = root_cli_parser.add_subparsers(
+        dest='check_name',
+        help='A check to be performed.',
+        required=True,
+    )
+    for subcommand_module in SUBCOMMAND_MODULES:
+        subcommand_parser = subcommand_parsers.add_parser(subcommand_module.CLI_SUBCOMMAND_NAME)
+        subcommand_parser.set_defaults(
+            invoke_cli_app=subcommand_module.invoke_cli_app,
+        )
+        subcommand_module.populate_argument_parser(subcommand_parser)
+
+
+def initialize_argument_parser() -> ArgumentParser:
+    """Return the root argument parser with sub-commands."""
+    root_cli_parser = ArgumentParser(prog=f'python -m {__package__ !s}')
+    attach_subcommand_parsers_to(root_cli_parser)
+    return root_cli_parser
+
+
+__all__ = ('initialize_argument_parser',)

src/pre_commit_terraform/_cli_subcommands.py

@@ -0,0 +1,12 @@
+"""A CLI sub-commands organization module."""
+
+from . import terraform_docs_replace
+from ._types import CLISubcommandModuleProtocol
+
+
+SUBCOMMAND_MODULES: list[CLISubcommandModuleProtocol] = [
+    terraform_docs_replace,
+]
+
+
+__all__ = ('SUBCOMMAND_MODULES',)

src/pre_commit_terraform/_errors.py

@@ -0,0 +1,16 @@
+"""App-specific exceptions."""
+
+
+class PreCommitTerraformBaseError(Exception):
+    """Base exception for all the in-app errors."""
+
+
+class PreCommitTerraformRuntimeError(
+        PreCommitTerraformBaseError,
+        RuntimeError,
+):
+    """An exception representing a runtime error condition."""
+
+
+class PreCommitTerraformExit(PreCommitTerraformBaseError, SystemExit):
+    """An exception for terminating execution from deep app layers."""

src/pre_commit_terraform/_structs.py

@@ -0,0 +1,16 @@
+"""Data structures to be reused across the app."""
+
+from enum import IntEnum
+
+
+class ReturnCode(IntEnum):
+    """POSIX-style return code values.
+
+    To be used in check callable implementations.
+    """
+
+    OK = 0
+    ERROR = 1
+
+
+__all__ = ('ReturnCode',)

src/pre_commit_terraform/_types.py

@@ -0,0 +1,30 @@
+"""Composite types for annotating in-project code."""
+
+from argparse import ArgumentParser, Namespace
+from typing import Final, Protocol
+
+from ._structs import ReturnCode
+
+
+ReturnCodeType = ReturnCode | int
+
+
+class CLISubcommandModuleProtocol(Protocol):
+    """A protocol for the subcommand-implementing module shape."""
+
+    CLI_SUBCOMMAND_NAME: Final[str]
+    """This constant contains a CLI."""
+
+    def populate_argument_parser(
+            self, subcommand_parser: ArgumentParser,
+    ) -> None:
+        """Run a module hook for populating the subcommand parser."""
+
+    def invoke_cli_app(
+            self, parsed_cli_args: Namespace,
+    ) -> ReturnCodeType | int:
+        """Run a module hook implementing the subcommand logic."""
+        ...  # pylint: disable=unnecessary-ellipsis
+
+
+__all__ = ('CLISubcommandModuleProtocol', 'ReturnCodeType')