DOC: user/explain reorg (and moving a lot of tutorials). (matplotlib#25395)

* DOC: make users/explain a sphinx gallery [ci doc]

This is a major re-org whereby much of galleries/tutorials is moved to galleries/user_explain, and hence users/explain in the published docs.  Much of this may be rearranged more over the 3.8 cycle.  

Additionally most tutorial references were moved from hard :doc: references to soft :ref: references for ease of further rearrangement.  

Note this uses the git head of sphinx-gallery and mpl-sphinx-theme.  We should watch the release cycle of sphinx-gallery so that it can be changed to one of their releases.

Author: jklymak

.circleci/config.yml

@@ -97,9 +97,12 @@ commands:
       - run:
           name: Install Python dependencies
           command: |
+            # remove when sphinx-gallery releases #1071:
             python -m pip install --user \
                 numpy<< parameters.numpy_version >> codecov coverage \
                 -r requirements/doc/doc-requirements.txt
+            python -m pip install --no-deps --user \
+                git+https://github.com/sphinx-gallery/sphinx-gallery.git
             python -m pip install --no-deps --user \
                 git+https://github.com/matplotlib/mpl-sphinx-theme.git
 

.flake8

@@ -55,22 +55,22 @@ per-file-ignores =
     lib/pylab.py: F401, F403
 
     doc/conf.py: E402
-    galleries/tutorials/advanced/path_tutorial.py: E402
-    galleries/tutorials/advanced/patheffects_guide.py: E402
-    galleries/tutorials/advanced/transforms_tutorial.py: E402, E501
-    galleries/tutorials/colors/colormaps.py: E501
-    galleries/tutorials/colors/colors.py: E402
-    galleries/tutorials/intermediate/artists.py: E402
-    galleries/tutorials/intermediate/constrainedlayout_guide.py: E402
-    galleries/tutorials/intermediate/legend_guide.py: E402
-    galleries/tutorials/intermediate/tight_layout_guide.py: E402
-    galleries/tutorials/introductory/animation_tutorial.py: E501
-    galleries/tutorials/introductory/images.py: E501
-    galleries/tutorials/introductory/pyplot.py: E402, E501
-    galleries/tutorials/text/annotations.py: E402, E501
-    galleries/tutorials/text/mathtext.py: E501
-    galleries/tutorials/text/text_intro.py: E402
-    galleries/tutorials/text/text_props.py: E501
+    galleries/users_explain/artists/paths.py: E402
+    galleries/users_explain/artists/patheffects_guide.py: E402
+    galleries/users_explain/artists/transforms_tutorial.py: E402, E501
+    galleries/users_explain/colors/colormaps.py: E501
+    galleries/users_explain/colors/colors.py: E402
+    galleries/tutorials/artists.py: E402
+    galleries/users_explain/axes/constrainedlayout_guide.py: E402
+    galleries/users_explain/axes/legend_guide.py: E402
+    galleries/users_explain/axes/tight_layout_guide.py: E402
+    galleries/users_explain/animations/animations.py: E501
+    galleries/tutorials/images.py: E501
+    galleries/tutorials/pyplot.py: E402, E501
+    galleries/users_explain/text/annotations.py: E402, E501
+    galleries/users_explain/text/mathtext.py: E501
+    galleries/users_explain/text/text_intro.py: E402
+    galleries/users_explain/text/text_props.py: E501
 
     galleries/examples/animation/frame_grabbing_sgskip.py: E402
     galleries/examples/images_contours_and_fields/tricontour_demo.py: E201

doc/Makefile

@@ -22,6 +22,7 @@ clean:
 	rm -rf "$(SOURCEDIR)/gallery"
 	rm -rf "$(SOURCEDIR)/plot_types"
 	rm -rf "$(SOURCEDIR)/tutorials"
+	rm -rf "$(SOURCEDIR)/users/explain"
 	rm -rf "$(SOURCEDIR)/savefig"
 	rm -rf "$(SOURCEDIR)/sphinxext/__pycache__"
 	rm -f $(SOURCEDIR)/_static/constrained_layout*.png

doc/api/index.rst

@@ -75,7 +75,7 @@ programmatic plot generation.
 Further reading:
 
 - The `matplotlib.pyplot` function reference
-- :doc:`/tutorials/introductory/pyplot`
+- :ref:`pyplot_tutorial`
 - :ref:`Pyplot examples <pyplots_examples>`
 
 .. _api-index:

doc/api/prev_api_changes/api_changes_1.4.x.rst

@@ -88,7 +88,7 @@ original location:
 * The legend handler interface has changed from a callable, to any object
   which implements the ``legend_artists`` method (a deprecation phase will
   see this interface be maintained for v1.4). See
-  :doc:`/tutorials/intermediate/legend_guide` for further details. Further legend changes
+  :ref:`legend_guide` for further details. Further legend changes
   include:
 
    * ``matplotlib.axes.Axes._get_legend_handles`` now returns a generator of

doc/api/prev_api_changes/api_changes_2.2.0.rst

@@ -9,7 +9,7 @@ New dependency
 
 `kiwisolver <https://github.com/nucleic/kiwi>`__ is now a required
 dependency to support the new constrained_layout,  see
-:doc:`/tutorials/intermediate/constrainedlayout_guide` for
+:ref:`constrainedlayout_guide` for
 more details.
 
 

doc/api/prev_api_changes/api_changes_3.4.0/behaviour.rst

@@ -24,7 +24,7 @@ library, so using it should be safe, but layouts may not be exactly the same as
 more development takes place.
 
 Details of using ``constrained_layout``, and its algorithm are available at
-:doc:`/tutorials/intermediate/constrainedlayout_guide`
+:ref:`constrainedlayout_guide`
 
 ``plt.subplot`` re-selection without keyword arguments
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

doc/api/style_api.rst

@@ -5,7 +5,7 @@
 Styles are predefined sets of `.rcParams` that define the visual appearance of
 a plot.
 
-:doc:`/tutorials/introductory/customizing` describes the mechanism and usage
+:ref:`customizing` describes the mechanism and usage
 of styles.
 
 The :doc:`/gallery/style_sheets/style_sheets_reference` gives an overview of

doc/api/toolkits/mplot3d.rst

@@ -12,7 +12,7 @@ and feel as regular 2D plots.  Not the fastest or most feature complete 3D
 library out there, but it ships with Matplotlib and thus may be a lighter
 weight solution for some use cases.
 
-See the :doc:`mplot3d tutorial </tutorials/toolkits/mplot3d>` for
+See the :ref:`mplot3d tutorial <mplot3d>` for
 more information.
 
 .. image:: /_static/demo_mplot3d.png

doc/conf.py

@@ -213,11 +213,14 @@ def matplotlib_reduced_latex_scraper(block, block_vars, gallery_conf,
         gallery_conf['image_srcset'] = []
     return matplotlib_scraper(block, block_vars, gallery_conf, **kwargs)
 
-gallery_dirs = [f'{ed}' for ed in ['gallery', 'tutorials', 'plot_types']
+gallery_dirs = [f'{ed}' for ed in
+                ['gallery', 'tutorials', 'plot_types', 'users/explain']
                 if f'{ed}/*' not in skip_subdirs]
 
-example_dirs = [f'../galleries/{gd}'.replace('gallery', 'examples')
-                for gd in gallery_dirs]
+example_dirs = []
+for gd in gallery_dirs:
+    gd = gd.replace('gallery', 'examples').replace('users/explain', 'users_explain')
+    example_dirs += [f'../galleries/{gd}']
 
 sphinx_gallery_conf = {
     'backreferences_dir': Path('api') / Path('_as_gen'),
@@ -244,6 +247,7 @@ def matplotlib_reduced_latex_scraper(block, block_vars, gallery_conf,
     'thumbnail_size': (320, 224),
     'within_subsection_order': gallery_order.subsectionorder,
     'capture_repr': (),
+    'copyfile_regex': r'.*\.rst',
 }
 
 if 'plot_gallery=0' in sys.argv: