Backport PR matplotlib#27948: Move IPython backend mapping to Matplotlib and support entry points

Author: QuLogic

.github/workflows/tests.yml

@@ -59,7 +59,8 @@ jobs:
             delete-font-cache: true
           - os: ubuntu-20.04
             python-version: 3.9
-            extra-requirements: '-r requirements/testing/extra.txt'
+            # One CI run tests ipython/matplotlib-inline before backend mapping moved to mpl
+            extra-requirements: '-r requirements/testing/extra.txt "ipython<8.24" "matplotlib-inline<0.1.7"'
             CFLAGS: "-fno-lto"  # Ensure that disabling LTO works.
             # https://github.com/matplotlib/matplotlib/pull/26052#issuecomment-1574595954
             # https://www.riverbankcomputing.com/pipermail/pyqt/2023-November/045606.html

doc/users/next_whats_new/backend_registry.rst

@@ -3,4 +3,15 @@ BackendRegistry
 
 New :class:`~matplotlib.backends.registry.BackendRegistry` class is the single
 source of truth for available backends. The singleton instance is
-``matplotlib.backends.backend_registry``.
+``matplotlib.backends.backend_registry``. It is used internally by Matplotlib,
+and also IPython (and therefore Jupyter) starting with IPython 8.24.0.
+
+There are three sources of backends: built-in (source code is within the
+Matplotlib repository), explicit ``module://some.backend`` syntax (backend is
+obtained by loading the module), or via an entry point (self-registering
+backend in an external package).
+
+To obtain a list of all registered backends use:
+
+    >>> from matplotlib.backends import backend_registry
+    >>> backend_registry.list_all()

galleries/users_explain/figure/backends.rst

@@ -175,7 +175,8 @@ QtAgg     Agg rendering in a Qt_ canvas (requires PyQt_ or `Qt for Python`_,
           more details.
 ipympl    Agg rendering embedded in a Jupyter widget (requires ipympl_).
           This backend can be enabled in a Jupyter notebook with
-          ``%matplotlib ipympl``.
+          ``%matplotlib ipympl`` or ``%matplotlib widget``.  Works with
+          Jupyter ``lab`` and ``notebook>=7``.
 GTK3Agg   Agg rendering to a GTK_ 3.x canvas (requires PyGObject_ and
           pycairo_).  This backend can be activated in IPython with
           ``%matplotlib gtk3``.
@@ -188,7 +189,8 @@ TkAgg     Agg rendering to a Tk_ canvas (requires TkInter_). This
           backend can be activated in IPython with ``%matplotlib tk``.
 nbAgg     Embed an interactive figure in a Jupyter classic notebook.  This
           backend can be enabled in Jupyter notebooks via
-          ``%matplotlib notebook``.
+          ``%matplotlib notebook`` or ``%matplotlib nbagg``.  Works with
+          Jupyter ``notebook<7`` and ``nbclassic``.
 WebAgg    On ``show()`` will start a tornado server with an interactive
           figure.
 GTK3Cairo Cairo rendering to a GTK_ 3.x canvas (requires PyGObject_ and
@@ -200,7 +202,7 @@ wxAgg     Agg rendering to a wxWidgets_ canvas (requires wxPython_ 4).
 ========= ================================================================
 
 .. note::
-   The names of builtin backends case-insensitive; e.g., 'QtAgg' and
+   The names of builtin backends are case-insensitive; e.g., 'QtAgg' and
    'qtagg' are equivalent.
 
 .. _`Anti-Grain Geometry`: http://agg.sourceforge.net/antigrain.com/
@@ -222,11 +224,13 @@ wxAgg     Agg rendering to a wxWidgets_ canvas (requires wxPython_ 4).
 .. _wxWidgets: https://www.wxwidgets.org/
 .. _ipympl: https://www.matplotlib.org/ipympl
 
+.. _ipympl_install:
+
 ipympl
 ^^^^^^
 
-The Jupyter widget ecosystem is moving too fast to support directly in
-Matplotlib.  To install ipympl:
+The ipympl backend is in a separate package that must be explicitly installed
+if you wish to use it, for example:
 
 .. code-block:: bash
 

galleries/users_explain/figure/figure_intro.rst

@@ -52,14 +52,20 @@ Notebooks and IDEs
 
 If you are using a Notebook (e.g. `Jupyter <https://jupyter.org>`_) or an IDE
 that renders Notebooks (PyCharm, VSCode, etc), then they have a backend that
-will render the Matplotlib Figure when a code cell is executed.  One thing to
-be aware of is that the default Jupyter backend (``%matplotlib inline``) will
+will render the Matplotlib Figure when a code cell is executed.  The default
+Jupyter backend (``%matplotlib inline``) creates static plots that
 by default trim or expand the figure size to have a tight box around Artists
-added to the Figure (see :ref:`saving_figures`, below).  If you use a backend
-other than the default "inline" backend, you will likely need to use an ipython
-"magic" like ``%matplotlib notebook`` for the Matplotlib :ref:`notebook
-<jupyter_notebooks_jupyterlab>` or ``%matplotlib widget`` for the  `ipympl
-<https://matplotlib.org/ipympl/>`_ backend.
+added to the Figure (see :ref:`saving_figures`, below).  For interactive plots
+in Jupyter you will need to use an ipython "magic" like ``%matplotlib widget``
+for the  `ipympl <https://matplotlib.org/ipympl/>`_ backend in ``jupyter lab``
+or ``notebook>=7``, or ``%matplotlib notebook`` for the Matplotlib
+:ref:`notebook <jupyter_notebooks_jupyterlab>` in ``notebook<7`` or
+``nbclassic``.
+
+.. note::
+
+    The  `ipympl <https://matplotlib.org/ipympl/>`_ backend is in a separate
+    package, see :ref:`Installing ipympl <ipympl_install>`.
 
 .. figure:: /_static/FigureNotebook.png
     :alt: Image of figure generated in Jupyter Notebook with notebook
@@ -75,15 +81,6 @@ other than the default "inline" backend, you will likely need to use an ipython
 .. seealso::
     :ref:`interactive_figures`.
 
-.. note::
-
-   If you only need to use the classic notebook (i.e. ``notebook<7``),
-   you can use:
-
-   .. sourcecode:: ipython
-
-   %matplotlib notebook
-
 .. _standalone-scripts-and-interactive-use:
 
 Standalone scripts and interactive use
@@ -104,7 +101,7 @@ backend.  These are typically chosen either in the user's :ref:`matplotlibrc
     QtAgg backend.
 
 When run from a script, or interactively (e.g. from an
-`iPython shell <https://ipython.readthedocs.io/en/stable/>`_) the Figure
+`IPython shell <https://ipython.readthedocs.io/en/stable/>`_) the Figure
 will not be shown until we call ``plt.show()``. The Figure will appear in
 a new GUI window, and usually will have a toolbar with Zoom, Pan, and other tools
 for interacting with the Figure.  By default, ``plt.show()`` blocks

galleries/users_explain/figure/writing_a_backend_pyplot_interface.rst

@@ -84,3 +84,47 @@ Function-based API
 2. **Showing figures**: `.pyplot.show()` calls a module-level ``show()``
    function, which is typically generated via the ``ShowBase`` class and its
    ``mainloop`` method.
+
+Registering a backend
+---------------------
+
+For a new backend to be usable via ``matplotlib.use()`` or IPython
+``%matplotlib`` magic command, it must be compatible with one of the three ways
+supported by the :class:`~matplotlib.backends.registry.BackendRegistry`:
+
+Built-in
+^^^^^^^^
+
+A backend built into Matplotlib must have its name and
+``FigureCanvas.required_interactive_framework`` hard-coded in the
+:class:`~matplotlib.backends.registry.BackendRegistry`.  If the backend module
+is not ``f"matplotlib.backends.backend_{backend_name.lower()}"`` then there
+must also be an entry in the ``BackendRegistry._name_to_module``.
+
+module:// syntax
+^^^^^^^^^^^^^^^^
+
+Any backend in a separate module (not built into Matplotlib) can be used by
+specifying the path to the module in the form ``module://some.backend.module``.
+An example is ``module://mplcairo.qt`` for
+`mplcairo <https:////github.com/matplotlib/mplcairo>`_.  The backend's
+interactive framework will be taken from its
+``FigureCanvas.required_interactive_framework``.
+
+Entry point
+^^^^^^^^^^^
+
+An external backend module can self-register as a backend using an
+``entry point`` in its ``pyproject.toml`` such as the one used by
+``matplotlib-inline``:
+
+.. code-block:: toml
+
+    [project.entry-points."matplotlib.backend"]
+    inline = "matplotlib_inline.backend_inline"
+
+The backend's interactive framework will be taken from its
+``FigureCanvas.required_interactive_framework``.  All entry points are loaded
+together but only when first needed, such as when a backend name is not
+recognised as a built-in backend, or when
+:meth:`~matplotlib.backends.registry.BackendRegistry.list_all` is first called.

lib/matplotlib/__init__.py

@@ -1208,14 +1208,16 @@ def use(backend, *, force=True):
         backend names, which are case-insensitive:
 
         - interactive backends:
-          GTK3Agg, GTK3Cairo, GTK4Agg, GTK4Cairo, MacOSX, nbAgg, QtAgg,
+          GTK3Agg, GTK3Cairo, GTK4Agg, GTK4Cairo, MacOSX, nbAgg, notebook, QtAgg,
           QtCairo, TkAgg, TkCairo, WebAgg, WX, WXAgg, WXCairo, Qt5Agg, Qt5Cairo
 
         - non-interactive backends:
           agg, cairo, pdf, pgf, ps, svg, template
 
         or a string of the form: ``module://my.module.name``.
 
+        notebook is a synonym for nbAgg.
+
         Switching to an interactive backend is not possible if an unrelated
         event loop has already been started (e.g., switching to GTK3Agg if a
         TkAgg window has already been opened).  Switching to a non-interactive

lib/matplotlib/backend_bases.py

@@ -1766,8 +1766,16 @@ def _fix_ipython_backend2gui(cls):
         # `ipython --auto`).  This cannot be done at import time due to
         # ordering issues, so we do it when creating a canvas, and should only
         # be done once per class (hence the `cache`).
-        if sys.modules.get("IPython") is None:
+
+        # This function will not be needed when Python 3.12, the latest version
+        # supported by IPython < 8.24, reaches end-of-life in late 2028.
+        # At that time this function can be made a no-op and deprecated.
+        mod_ipython = sys.modules.get("IPython")
+        if mod_ipython is None or mod_ipython.version_info[:2] >= (8, 24):
+            # Use of backend2gui is not needed for IPython >= 8.24 as the
+            # functionality has been moved to Matplotlib.
             return
+
         import IPython
         ip = IPython.get_ipython()
         if not ip:
@@ -2030,9 +2038,8 @@ def _switch_canvas_and_return_print_method(self, fmt, backend=None):
         canvas = None
         if backend is not None:
             # Return a specific canvas class, if requested.
-            canvas_class = (
-                importlib.import_module(cbook._backend_module_name(backend))
-                .FigureCanvas)
+            from .backends.registry import backend_registry
+            canvas_class = backend_registry.load_backend_module(backend).FigureCanvas
             if not hasattr(canvas_class, f"print_{fmt}"):
                 raise ValueError(
                     f"The {backend!r} backend does not support {fmt} output")