Overhaul Manage Add-ons and Packages (#1724)

stevepiercy · web-flow · commit 5f879e601901 · 2024-10-02T00:52:53.000-07:00
* Overhaul Manage Add-ons and Packages
- Create a new Diataxis explanation section, Conceptual guides.
- Create a new section, Manage Plone, and pages for frontend and backend.
- Move bits from manage-add-ons-packages.md into appropriate files

* Add a redirect

* Move cookiecutter info to Glossary

* Move Configuration with `cookiecutter-zope-instance` to manage/backend.md

* Remove manage-add-ons-packages.md
diff --git a/docs/_static/conceptual-guides/xkcd-1987-python-environment.png b/docs/_static/conceptual-guides/xkcd-1987-python-environment.png
diff --git a/docs/conceptual-guides/index.md b/docs/conceptual-guides/index.md
@@ -0,0 +1,20 @@
+---
+myst:
+  html_meta:
+    "description": "Conceptual guides provide explanation of concepts to deepen and broaden your understanding of Plone."
+    "property=og:description": "Conceptual guides provide explanation of concepts to deepen and broaden your understanding of Plone."
+    "property=og:title": "Conceptual guides"
+    "keywords": "Plone 6, Conceptual guides"
+---
+
+# Conceptual guides
+
+This part of the documentation provides explanation of concepts to deepen and broaden your understanding of Plone.
+
+
+```{toctree}
+:maxdepth: 2
+
+package-management
+make-build-backend-walk-through
+```
diff --git a/docs/conceptual-guides/make-build-backend-walk-through.md b/docs/conceptual-guides/make-build-backend-walk-through.md
@@ -0,0 +1,58 @@
+---
+myst:
+  html_meta:
+    "description": "make build-backend in Plone."
+    "property=og:description": "make build-backend in Plone."
+    "property=og:title": "make build-backend details"
+    "keywords": "Plone 6, make, build-backend"
+---
+
+(make-build-backend-details-label)=
+
+# `make build-backend` details
+
+This chapter assumes you have previously followed {doc}`/install/create-project`.
+
+The `Makefile` at the root of your project invokes commands in `backend/Makefile`.
+Here are excerpts from `backend/Makefile` to show details of the `make build-backend` command.
+
+```makefile
+bin/pip:
+	@echo "$(GREEN)==> Setup Virtual Env$(RESET)"
+	python3 -m venv .
+	bin/pip install -U "pip" "wheel" "cookiecutter" "mxdev"
+
+instance/etc/zope.ini:	bin/pip
+	@echo "$(GREEN)==> Install Plone and create instance$(RESET)"
+	bin/cookiecutter -f --no-input --config-file instance.yaml https://github.com/plone/cookiecutter-zope-instance
+	mkdir -p var/{filestorage,blobstorage,cache,log}
+
+# ...
+
+.PHONY: build-dev
+build-dev: instance/etc/zope.ini ## pip install Plone packages
+	@echo "$(GREEN)==> Setup Build$(RESET)"
+	bin/mxdev -c mx.ini
+	bin/pip install -r requirements-mxdev.txt
+```
+
+The command `make build-backend`:
+
+-   Invokes the target `build-dev` target in `backend/Makefile`.
+-   This invokes the target `instance/etc/zope.ini`.
+-   This invokes the target `bin/pip`.
+
+    -   This creates a `Python` virtual environment if one does not exist.
+    -   It installs and upgrades Python package management tools in that virtual environment.
+
+-   Returning to the target `instance/etc/zope.ini`:
+
+    -   This creates or updates the Zope configuration from its `instance.yaml` file using `cookiecutter-zope-instance`.
+    -   Creates specified directories, if they do not exist.
+
+-   Returning to the target `build-dev`:
+
+    -   This generates the `mxdev` files as described above in {ref}`manage-mxdev-usage-overview-label`.
+    -   Installs Plone core packages and add-ons from the files generated by `mxdev`.
+
+You can configure your Zope instance as described in the section {ref}`manage-common-management-tasks-label`.
diff --git a/docs/conceptual-guides/package-management.md b/docs/conceptual-guides/package-management.md
@@ -0,0 +1,64 @@
+---
+myst:
+  html_meta:
+    "description": "Package management in Plone."
+    "property=og:description": "Package management in Plone."
+    "property=og:title": "Package management"
+    "keywords": "Plone 6, package management, mxdev"
+---
+
+# Package management
+
+Plone 6 consists of a collection of Python and Node.js packages.
+Over the decades of its existence, Plone has used several package management tools, sometimes multiple tools at one time.
+Each one has its strengths and weaknesses for performing specific tasks, such as installation, conflict resolution, updates and upgrades, and working with virtual environments and across platforms.
+
+With Volto as the default frontend in Plone 6, first npm, then pnpm, was brought into the mix as a package manager for its Node.js packages.
+
+Python itself has a complex and convoluted history with package management, as [xkcd](https://xkcd.com/1987/) illustrates.
+
+```{image} /_static/conceptual-guides/xkcd-1987-python-environment.png
+:alt: A comic from xkcd entitled Python Environment
+:class: figure
+:target: https://xkcd.com/1987/
+```
+
+
+(manage-backend-python-packages-label)=
+
+## Manage backend Python packages
+
+If you want to check out a Plone core package for development, or want to override the constraints of Plone, normally you would define constraints with a file {file}`constraints.txt` to tell `pip` to install a different version of a Plone package.
+
+```text
+# constraints.txt with unresolvable version conflict
+-c https://dist.plone.org/release/{PLONE_BACKEND_PATCH_VERSION}/constraints.txt
+plone.api>=2.0.0a3
+```
+
+Unfortunately `pip` does not allow overriding constraints this way. 
+{term}`mxdev` solves this issue.
+
+
+### `mxdev` to the rescue!
+
+`mxdev` resolves Plone constraints according to your needs for pinning versions or source checkouts.
+It reads its configuration file {file}`mx.ini`, and your {file}`requirements.txt` and {file}`constraints.txt` files.
+Then it fetches the requirements and constraints of Plone.
+Finally, it writes new combined requirements in {file}`requirements-mxdev.txt` and new constraints in {file}`constraints-mxdev.txt`.
+Together these two files contain the combined requirements and constraints, but modified according to the configuration in {file}`mx.ini`.
+The generated files indicate from where the constraints were fetched, and comments are added when a modification was necessary.
+
+`mxdev` does not run `pip` or install packages.
+You or your development tools, such as GNU Make, must perform that step.
+
+```{seealso}
+{doc}`/manage/backend`
+```
+
+
+## Manage frontend Node.js packages
+
+```{todo}
+Why do we use pnpm?
+```
diff --git a/docs/conf.py b/docs/conf.py
@@ -245,6 +245,7 @@
     "contributing/plone-restapi": "/plone.restapi/docs/source/contributing/index.html",
     "contributing/volto": "/volto/contributing/index.html",
     "install/install-from-packages": "/install/create-project.html",
+    "manage/frontend": "/volto/addons/index.html",
 }
 
 
diff --git a/docs/glossary.md b/docs/glossary.md
@@ -48,7 +48,9 @@ plone/generator-volto
     See {ref}`upgrade-18-cookieplone-label`.
 
 cookiecutter-plone-starter
-    [cookiecutter-plone-starter](https://github.com/collective/cookiecutter-plone-starter/) is a framework for jumpstarting Plone 6 projects quickly.
+    [cookiecutter-plone-starter](https://github.com/collective/cookiecutter-plone-starter/) creates a Plone project that you can install using {term}`Make`.
+    It generates files for installing and configuring both the frontend and backend.
+    For the backend, it uses {term}`cookiecutter-zope-instance` to generate configuration files for a Zope WSGI instance.
 
 cookiecutter-zope-instance
     [cookiecutter-zope-instance](https://github.com/plone/cookiecutter-zope-instance) is a cookiecutter template to create a full and complex configuration of a Zope WSGI instance.
diff --git a/docs/index.md b/docs/index.md
@@ -32,13 +32,15 @@ Read the [documentation for the previous version, Plone 5](https://5.docs.plone.
 
 overview/index
 install/index
+manage/index
 upgrade/index
 deployment/index
 volto/index
 plone.restapi/docs/source/index
 backend/index
 classic-ui/index
 i18n-l10n/index
+conceptual-guides/index
 contributing/index
 ```
 
diff --git a/docs/install/index.md b/docs/install/index.md
@@ -55,6 +55,5 @@ Choose a version to demo.
 create-project
 create-project-classic-ui
 create-project-cookieplone
-manage-add-ons-packages
 containers/index
 ```
diff --git a/docs/manage/backend.md b/docs/manage/backend.md
@@ -1,90 +1,30 @@
 ---
 myst:
   html_meta:
-    "description": "Manage add-ons, packages, and processes"
-    "property=og:description": "Manage add-ons, packages, and processes"
-    "property=og:title": "Manage add-ons, packages, and processes"
-    "keywords": "Plone 6, manage, backend, add-ons, packages, processes, cookiecutter, Zope"
+    "description": "Manage Plone backend"
+    "property=og:description": "Manage Plone backend"
+    "property=og:title": "Manage Plone backend"
+    "keywords": "Plone 6, manage, backend, add-ons, packages, mxdev"
 ---
 
+(manage-plone-backend-label)=
 
-(manage-add-ons-packages-and-processes-label)=
+# Manage Plone backend
 
-# Manage add-ons and packages
+This part of the documentation describes how to perform common management tasks in the Plone backend.
+This chapter assumes you have previously followed {doc}`/install/create-project`.
 
-This chapter assumes you have previously followed {doc}`create-project`.
-In this section, we discuss details of the installation process so that you can customize your Plone installation.
-It also covers routine management tasks that a developer might perform.
 
+## Manage add-ons and packages
 
-(manage-installation-details-with-cookiecutter-label)=
-
-## Installation details with Cookiecutter
-
-{term}`Cookiecutter` creates projects from project templates.
-The cookiecutter [`cookiecutter-plone-starter`](https://github.com/collective/cookiecutter-plone-starter/) creates a Plone project that you can install using {term}`Make`.
-It generates files for installing and configuring both the frontend and backend.
-For the backend, it uses [`cookiecutter-zope-instance`](https://github.com/plone/cookiecutter-zope-instance) to generate configuration files for a Zope WSGI instance.
-
-
-(manage-configuration-with-cookiecutter-zope-instance-label)=
-
-## Configuration with `cookiecutter-zope-instance`
-
-You can configure your instance's options, including the following.
-
--   persistent storage: blobs, direct filestorage, relational database, ZEO, and so on
--   ports
--   threads
--   cache
--   debugging and profiling for development
+Plone uses `mxdev` to manage packages and constraints.
 
 ```{seealso}
-For a complete list of features, usage, and options, read [`cookiecutter-zope-instance`'s `README.rst`](https://github.com/plone/cookiecutter-zope-instance#readme).
+For an explanation of why Plone uses `mxdev`, see {ref}`manage-backend-python-packages-label`.
 ```
 
 
-(manage-plone-backend-packages-with-mxdev-label)=
-
-## Manage Plone backend packages with `mxdev`
-
-This section describes how to manage packages for the Plone backend with `mxdev`.
-
-For developing add-ons for the Plone frontend, Volto, see {doc}`volto/addons/index`.
-
-
-(manage-the-problem-with-pip-label)=
-
-### The problem with `pip`
-
-If you want to check out a Plone core package for development, or want to override the constraints of Plone, normally you would define constraints with a file {file}`constraints.txt` to tell `pip` to install a different version of a Plone package.
-
-```
-# constraints.txt with unresolvable version conflict
--c https://dist.plone.org/release/{PLONE_BACKEND_PATCH_VERSION}/constraints.txt
-plone.api>=2.0.0a3
-```
-
-Unfortunately `pip` does not allow overriding constraints this way. 
-{term}`mxdev` solves this issue.
-
-
-(manage-mxdev-to-the-rescue-label)=
-
-### `mxdev` to the rescue!
-
-`mxdev` resolves Plone constraints according to your needs for version pinning or source checkouts.
-It reads its configuration file {file}`mx.ini`, and your {file}`requirements.txt` and {file}`constraints.txt` files.
-Then it fetches the requirements and constraints of Plone.
-Finally, it writes new combined requirements in {file}`requirements-mxdev.txt` and new constraints in {file}`constraints-mxdev.txt`.
-Together these two files contain the combined requirements and constraints, but modified according to the configuration in {file}`mx.ini`.
-The generated files indicate from where the constraints were fetched, and comments are added when a modification was necessary.
-
-`mxdev` does not run `pip` or install packages.
-You must perform that step.
-
-
-(manage-mxdev-usage-overview-label)=
+(mxdev-usage-overview-label)=
 
 ### `mxdev` usage overview
 
@@ -140,7 +80,7 @@ make build-backend
 `make build-backend` invokes `mxdev`, which generates the files {file}`requirements-mxdev.txt` and {file}`constraints-mxdev.txt`.
 It then invokes `pip` to install packages with the new requirements file.
 
-To reload the packages, stop your Zope instance/Plone site with {kbd}`ctrl-c`, and start it with the following command.
+To reload the packages, stop your Plone site with {kbd}`ctrl-c`, and start it with the following command.
 
 ```shell
 make start-backend
@@ -151,15 +91,6 @@ See the [documentation of `mxdev` in its README.md](https://github.com/mxstack/m
 ```
 
 
-(manage-common-management-tasks-label)=
-
-## Common management tasks
-
-This section provides examples of common add-on and package management tasks.
-For the examples, we will modify the default files from the previous section {ref}`manage-mxdev-usage-overview-label`.
-We will also use Classic UI for the interface because some packages and add-ons have not yet been updated to work with the new frontend.
-
-
 (manage-add-an-add-on)=
 
 ### Add an add-on
@@ -346,51 +277,23 @@ In a web browser, visit http://localhost:8080/ to see that Plone is running.
 
 Your instance is running in the foreground.
 
-
-(manage-installation-details-label)=
-
-## Backend installation details
-
-The `Makefile` at the root of your project invokes commands in `backend/Makefile`.
-Here are excerpts from `backend/Makefile` to show details of the `make build-backend` command.
-
-```makefile
-bin/pip:
-	@echo "$(GREEN)==> Setup Virtual Env$(RESET)"
-	python3 -m venv .
-	bin/pip install -U "pip" "wheel" "cookiecutter" "mxdev"
-
-instance/etc/zope.ini:	bin/pip
-	@echo "$(GREEN)==> Install Plone and create instance$(RESET)"
-	bin/cookiecutter -f --no-input --config-file instance.yaml https://github.com/plone/cookiecutter-zope-instance
-	mkdir -p var/{filestorage,blobstorage,cache,log}
-
-# ...
-
-.PHONY: build-dev
-build-dev: instance/etc/zope.ini ## pip install Plone packages
-	@echo "$(GREEN)==> Setup Build$(RESET)"
-	bin/mxdev -c mx.ini
-	bin/pip install -r requirements-mxdev.txt
+```{seealso}
+For an explanation of the command `make build-backend`, see {doc}`/conceptual-guides/make-build-backend-walk-through`.
 ```
 
-The command `make build-backend`:
-
--   Invokes the target `build-dev` target in `backend/Makefile`.
--   This invokes the target `instance/etc/zope.ini`.
--   This invokes the target `bin/pip`.
-
-    -   This creates a `Python` virtual environment if one does not exist.
-    -   It installs and upgrades Python package management tools in that virtual environment.
 
--   Returning to the target `instance/etc/zope.ini`:
+(manage-configuration-with-cookiecutter-zope-instance-label)=
 
-    -   This creates or updates the Zope configuration from its `instance.yaml` file using `cookiecutter-zope-instance`.
-    -   Creates specified directories, if they do not exist.
+## Configuration with `cookiecutter-zope-instance`
 
--   Returning to the target `build-dev`:
+You can configure your instance's options, including the following.
 
-    -   This generates the `mxdev` files as described above in {ref}`manage-mxdev-usage-overview-label`.
-    -   Installs Plone core packages and add-ons from the files generated by `mxdev`.
+-   persistent storage: blobs, direct filestorage, relational database, ZEO, and so on
+-   ports
+-   threads
+-   cache
+-   debugging and profiling for development
 
-You can configure your Zope instance as described in the section {ref}`manage-common-management-tasks-label`.
+```{seealso}
+For a complete list of features, usage, and options, read [`cookiecutter-zope-instance`'s `README.rst`](https://github.com/plone/cookiecutter-zope-instance#readme).
+```
diff --git a/docs/manage/frontend.md b/docs/manage/frontend.md
@@ -0,0 +1,18 @@
+---
+myst:
+  html_meta:
+    "description": "Manage Plone frontend"
+    "property=og:description": "Manage Plone frontend"
+    "property=og:title": "Manage Plone frontend"
+    "keywords": "Plone 6, manage, frontend, add-ons, packages"
+---
+
+(manage-plone-frontend-label)=
+
+# Manage Plone frontend
+
+This part of the documentation describes how to perform common management tasks in the Plone frontend.
+
+```{seealso}
+{doc}`/volto/addons/index`
+```
diff --git a/docs/manage/index.md b/docs/manage/index.md

Original file line number	Diff line number	Diff line change
`@@ -245,6 +245,7 @@`
`245`	`245`	`"contributing/plone-restapi": "/plone.restapi/docs/source/contributing/index.html",`
`246`	`246`	`"contributing/volto": "/volto/contributing/index.html",`
`247`	`247`	`"install/install-from-packages": "/install/create-project.html",`
	`248`	`+ "manage/frontend": "/volto/addons/index.html",`
`248`	`249`	`}`
`249`	`250`
`250`	`251`