add logical plugin example to tutorial fix #122

Author: bckohan

django_typer/management/__init__.py

@@ -1951,6 +1951,7 @@ def __new__(
                         attr_help = base.help
 
             def command_bases() -> t.Generator[t.Type[TyperCommand], None, None]:
+                # the mro is not yet resolved so we have to do it manually
                 seen = set()
                 for first_level in reversed(bases):
                     for base in reversed(first_level.__mro__):

doc/source/changelog.rst

@@ -2,14 +2,15 @@
 Change Log
 ==========
 
-v2.2.3 (2024-10-xx)
+v2.3.0 (2024-10-xx)
 ===================
 
 * Implemented `Drop python 3.8 support. <https://github.com/django-commons/django-typer/issues/130>`_
 * Implemented `Command help order should respect definition order for class based commands. <https://github.com/django-commons/django-typer/issues/129>`_
 * Fixed `Overriding the command group class does not work. <https://github.com/django-commons/django-typer/issues/128>`_
 * Completed `Add project to test PyPI <https://github.com/django-commons/django-typer/issues/126>`_
 * Completed `Open up vulnerability reporting and add security policy. <https://github.com/django-commons/django-typer/issues/124>`_
+* Completed `Add example of custom plugin logic to plugins tutorial. <https://github.com/django-commons/django-typer/issues/122>`_
 * Completed `Move architecture in docs to ARCHITECTURE.md <https://github.com/django-commons/django-typer/issues/121>`_
 * Completed `Transfer to django-commons <https://github.com/django-commons/django-typer/issues/117>`_
 * Completed `Add howto for how to change the display order of commands in help. <https://github.com/django-commons/django-typer/issues/116>`_

doc/source/extensions.rst

@@ -6,17 +6,27 @@
 Tutorial: Inheritance & Plugins
 ===============================
 
-You may need to change the behavior of an
-`upstream command <https://en.wikipedia.org/wiki/Upstream_(software_development)>`_ or wish
-you could add an additional subcommand or group to it. django-typer_ offers two patterns for
-changing or extending the behavior of commands. :class:`~django_typer.management.TyperCommand`
-classes :ref:`support inheritance <inheritance>`, even multiple inheritance. This can be a way to
-override or add additional commands to a command implemented elsewhere. You can then use Django's
-built in command override precedence (INSTALLED_APPS) to ensure your command is used instead of the
-upstream command or give it a different name if you would like the upstream command to still be
-available. The :ref:`plugin pattern <plugin>` allows commands and groups to be added or overridden
-directly on upstream commands without inheritance. This mechanism is useful when you might expect
-other apps to also modify the original command. Conflicts are resolved in INSTALLED_APPS order.
+Adding to, or altering the behavior of, commands from upstream Django_ apps is a common use case.
+Doing so allows you to keep your CLI_ stable while adding additional behaviors through your site's
+configuration settings files. There are three main extension patterns you may wish to employ:
+
+    1. Override the behavior of a command in an upstream app.
+    2. Add additional subcommands or groups to a command in an upstream app.
+    3. Hook implementations of custom logic into upstream command extension points.
+       (`Inversion of Control <https://en.wikipedia.org/wiki/Inversion_of_control>`_)
+
+The django-typer_ plugin mechanism supports all three of these use cases in a way that respects
+the precedence order of apps in the ``INSTALLED_APPS`` setting. In this tutorial we walk through
+an example of each using a :ref:`generic backup command <generic_backup>`. First we'll see how we
+might  :ref:`use inheritance (1) <inheritance>` to override and change the behavior of a
+subcommand. Then we'll see how we can :ref:`add subcommands (2) <plugin>` to an upstream command
+using plugins. Finally we'll use pluggy_ to implement a hook system that allows us to
+:ref:`add custom logic (3) <hooks>` to an upstream command.
+
+.. _generic_backup:
+
+A Generic Backup Command
+-------------------------
 
 Consider the task of backing up a Django website. State is stored in the database, in media files
 on disk, potentially in other files, and also in the software stack running on the server. If we
@@ -68,8 +78,8 @@ Inheritance
 -----------
 
 The first option we have is simple inheritance. Lets say the base command is defined in
-an app called backup. Now say we have another app that adds functionality that uses media
-files to our site. This means we'll want to add a media backup routine to the backup command.
+an app called backup. Now say we have another app that uses media files. This means we'll
+want to add a media backup routine to the backup command.
 
 .. note::
 
@@ -162,17 +172,14 @@ backup batch:
     [.............................................]
     Backing up ./media to ./media.tar.gz
 
-.. warning::
-
-    Inheritance is not supported from typical Django commands that used argparse to define their
-    interface.
 
+.. _inheritance_rationale:
 
 When Does Inheritance Make Sense?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Inheritance is a good choice when you want to tweak the behavior of a specific command and do not
-expect other apps to also modify the same command. It's also a good choice when you want to offer
+Inheritance is a good choice when you want to tweak the behavior of a specific command and **do not
+expect other apps to also modify the same command**. It's also a good choice when you want to offer
 a different flavor of a command under a different name.
 
 What if other apps want to alter the same command and we don't know about them, but they may end up
@@ -181,7 +188,7 @@ installed along with our app? This is where the plugin pattern will serve us bet
 
 .. _plugin:
 
-Plugins
+CLI Plugins
 -----------
 
 **The plugin pattern allows us to add or override commands and groups on an upstream command
@@ -194,10 +201,10 @@ than ``commands``. Let us suppose we are developing a site that uses the backup
 upstream and we've implemented most of our custom site functionality in a new app called my_app.
 Because we're now mostly working at the level of our particular site we may want to add more custom
 backup logic. For instance, lets say we know our site will always run on sqlite and we prefer
-to just copy the file to backup our database. Lets also pretend that it is useful for us to backup
-the python stack (e.g. requirements.txt) running on our server. To do that we can use the
-plugin pattern to add our environment backup routine and override the database routine from
-the upstream backup app. Our app tree now might look like this:
+to just copy the file to backup our database. It is also useful for us to capture the python stack
+(e.g. requirements.txt) running on our server. To do that we can use the plugin pattern to add our
+environment backup routine and override the database routine from the upstream backup app. Our app
+tree now might look like this:
 
 .. code-block:: text
 
@@ -230,10 +237,9 @@ the upstream backup app. Our app tree now might look like this:
                 └── backup.py
 
 
-Note that we've added an ``plugins`` directory to the management directory of the media and
-my_app apps. This is where we'll place our extension commands. There is an additional step we must
-take. In the ``apps.py`` file of the media and my_app apps we must register our plugins like
-this:
+Note that we've added a ``plugins`` directory to the management directory of the ``media`` and
+``my_app`` apps. This is where we'll place our command extensions. We must register our plugins
+directory in the ``apps.py`` file of the media and my_app apps like this:
 
 .. code-block:: python
 
@@ -253,14 +259,14 @@ this:
     Because we explicitly register our plugins we can call the package whatever we want.
     django-typer does not require it to be named ``plugins``. It is also important to
     do this inside ready() because conflicts are resolved in the order in which the extension
-    modules are registered and ready() methods are called in INSTALLED_APPS order.
+    modules are registered and ready() methods are called in ``INSTALLED_APPS`` order.
 
-For plugins to work, we'll need to re-mplement media from above as a composed extension
-and that would look like this:
+For plugins to work, we'll need to re-implement media from above as a composed extension
+like this:
 
 .. tabs::
 
-    .. tab:: django-typer
+    .. tab:: Django-style
 
         .. literalinclude:: ../../tests/apps/examples/plugins/media2/management/plugins/backup.py
             :language: python
@@ -425,8 +431,10 @@ You may even override the initializer of a predefined group:
     (i.e. ``Command.grp1.grp2.grp3.cmd``), if there is only one cmd you can simply write
     ``Command.cmd``. However, using the strict hierarchy will be robust to future changes.
 
-When Do Plugins Make Sense?
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.. _cli_plugin_rationale:
+
+When Do CLI Plugins Make Sense?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Plugins can be used to group like behavior together under a common root command. This can be
 thought of as a way to namespace CLI tools or easily share significant code between tools that have
@@ -441,3 +449,124 @@ Plugins can be a good way to organize commands in a code base that follows this
 also allows for deployments that install a subset of those apps and is therefore a good way to
 organize commands in code bases that serve as a framework for a particular kind of site or that
 support selecting the features to install by the inclusion or exclusion of specific apps.
+
+
+.. _hooks:
+
+Logic Plugins
+-------------
+
+`Inversion of Control (IoC) <https://en.wikipedia.org/wiki/Inversion_of_control>`_ is a design
+pattern that allows you to inject custom logic into a framework or library. The framework defines
+the general execution flow with extension points along the way that downstream applications can
+provide the implementations for. Django uses IoC all over the place. Extension points are often
+called ``hooks``. **You may use a third party library to manage hooks or implement your own
+mechanism but you will always need to register hook implementations. The same plugin mechanism we
+used in the** :ref:`last section <plugin>` **provides a natural place to do this.**
+
+Some Django_ apps may keep state in files in places on the filesystem unknown to other parts of
+your code base. In this section we'll use pluggy_ to define a hook for other apps to implement to
+backup their own files. Let's:
+
+1. Create a new app ``backup_files`` and inherit from our the extended media backup command we
+   created in the :ref:`inheritance section <inheritance>`.
+2. Define a pluggy_ interface for backing up arbitrary files
+3. Add a ``files`` command to our backup command that will call all registered
+   hooks to backup their own files.
+
+.. tabs::
+
+    .. tab:: Django-style
+
+        .. literalinclude:: ../../tests/apps/examples/plugins/backup_files/management/commands/backup.py
+            :language: python
+            :caption: backup_files/management/commands/backup.py
+            :linenos:
+            :replace:
+                tests.apps.examples.plugins.media1: media 
+
+    .. tab:: Typer-style
+
+        .. literalinclude:: ../../tests/apps/examples/plugins/backup_files/management/commands/backup_typer.py
+            :language: python
+            :caption: backup_files/management/commands/backup.py
+            :linenos:
+            :replace:
+                tests.apps.examples.plugins.media1: media
+
+Now lets define two new apps, files1 and files2 that will provide and register implementations of
+the backup_files hook:
+
+.. tabs::
+
+    .. tab:: Django-style
+
+        .. literalinclude:: ../../tests/apps/examples/plugins/files1/management/plugins/backup.py
+            :language: python
+            :caption: files1/management/plugins/backup.py
+            :linenos:
+            :replace:
+                tests.apps.examples.plugins.backup_files: backup_files
+
+    .. tab:: Typer-style
+
+        .. literalinclude:: ../../tests/apps/examples/plugins/files1/management/plugins/backup_typer.py
+            :language: python
+            :caption: files1/management/plugins/backup.py
+            :linenos:
+            :replace:
+                tests.apps.examples.plugins.backup_files: backup_files
+
+.. tabs::
+
+    .. tab:: Django-style
+
+        .. literalinclude:: ../../tests/apps/examples/plugins/files2/management/plugins/backup.py
+            :language: python
+            :caption: files2/management/plugins/backup.py
+            :linenos:
+            :replace:
+                tests.apps.examples.plugins.backup_files: backup_files
+
+    .. tab:: Typer-style
+
+        .. literalinclude:: ../../tests/apps/examples/plugins/files2/management/plugins/backup_typer.py
+            :language: python
+            :caption: files2/management/plugins/backup.py
+            :linenos:
+            :replace:
+                tests.apps.examples.plugins.backup_files: backup_files
+
+Both ``files1`` and ``files2`` will need to register their plugin packages in their ``apps.py``
+file:
+
+.. literalinclude:: ../../tests/apps/examples/plugins/files1/apps.py
+    :language: python
+    :caption: files1/apps.py
+    :linenos:
+    :replace:
+        tests.apps.examples.plugins.files1: files1
+
+Now when we run we see:
+
+.. code-block:: bash
+
+    $> python manage.py backup
+    Backing up database [default] to: ./default.json
+    [.............................................]
+    Backing up ./media to ./media.tar.gz
+    Backed up files to ./files2.zip
+    Backed up files to ./files1.tar.gz
+
+
+When Do Logic Plugins Make Sense?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:ref:`CLI plugins make sense <cli_plugin_rationale>` when you want to add additional commands or
+under a common namespace or to override the entire behavior of a command. Logical plugins make
+more sense in the weeds of a particular subroutine. Our example above has the following qualities
+which makes it a good candidate:
+
+1. The logic makes sense under a common root name (e.g. ``./manage.py backup files``).
+2. Multiple apps may need to execute their own version of the logic to complete the operation.
+3. The logic is amenable to a common interface that all plugins can implement.

doc/source/refs.rst

@@ -17,3 +17,5 @@
 .. _Options: https://typer.tiangolo.com/tutorial/options/
 .. _call_command: https://docs.djangoproject.com/en/5.0/ref/django-admin/#running-management-commands-from-your-code
 .. _sphinxcontrib-typer: https://pypi.org/project/sphinxcontrib-typer/
+.. _pluggy: https://pluggy.readthedocs.io/
+.. _CLI: https://en.wikipedia.org/wiki/Command-line_interface

examples/plugins/backup/management/commands/backup_typer.py

@@ -12,6 +12,10 @@
 
 app = Typer()
 
+# these two lines are not necessary but will make your type checker happy
+assert app.django_command
+Command = app.django_command
+
 Command.suppressed_base_arguments = {"verbosity", "skip_checks"}
 Command.requires_migrations_checks = False
 Command.requires_system_checks = []

examples/plugins/backup_files/__init__.py

@@ -0,0 +1,8 @@
+from django.apps import AppConfig
+
+from django_typer.utils import register_command_plugins
+
+
+class BackupFilesConfig(AppConfig):
+    name = "tests.apps.examples.plugins.backup_files"
+    label = name.replace(".", "_")

examples/plugins/backup_files/apps.py

@@ -0,0 +1,41 @@
+import sys
+import typing as t
+from pathlib import Path
+
+import typer
+import pluggy
+
+from tests.apps.examples.plugins.media1.management.commands.backup import (
+    Command as Backup,
+)
+
+
+class Command(Backup):  # inherit from the extended media backup command
+    plugins = pluggy.PluginManager("backup")
+    hookspec = pluggy.HookspecMarker("backup")
+    hookimpl = pluggy.HookimplMarker("backup")
+
+    # add a new command called files that delegates file backups to plugins
+    @Backup.command()
+    def files(self):
+        """
+        Backup app specific non-media files.
+        """
+        for archive in self.plugins.hook.backup_files(command=self):
+            if archive:
+                typer.echo(f"Backed up files to {archive}")
+
+
+@Command.hookspec
+def backup_files(command: Command) -> t.Optional[Path]:
+    """
+    A hook for backing up app specific files.
+
+    Must return the path to the archive file or None if no files were backed up.
+
+    :param command: the backup command instance
+    :return: The path to the archived backup file
+    """
+
+
+Command.plugins.add_hookspecs(sys.modules[__name__])