Merge pull request #62 from DiamondLightSource/dev

Architectural Decisions Records

Author: gilesknap

README.rst

@@ -3,10 +3,9 @@ python3-pip-skeleton-cli
 
 |code_ci| |docs_ci| |coverage| |pypi_version| |license|
 
-This skeleton module (inspired by `jaraco/skeleton
-<https://blog.jaraco.com/skeleton/>`_) is a generic Python project structure
-which provides a means to keep tools and techniques in sync between multiple
-Python projects.
+``python3-pip-skeleton-cli`` provides the documentation
+and a command line tool to enable the adoption of python3-pip-skeleton_
+into a new or existing Python project.
 
 ============== ==============================================================
 PyPI           ``pip install python3-pip-skeleton``
@@ -15,7 +14,16 @@ Documentation  https://DiamondLightSource.github.io/python3-pip-skeleton-cli
 Releases       https://github.com/DiamondLightSource/python3-pip-skeleton-cli/releases
 ============== ==============================================================
 
-It integrates the following tools:
+The related python3-pip-skeleton_ repository contains the source
+code that can be merged into new or existing projects, and pulled from to
+keep them up to date. It can also serve as a working example for those who
+would prefer to cherry-pick.
+
+python3-pip-skeleton_ is inspired by `jaraco/skeleton
+<https://blog.jaraco.com/skeleton/>`_.
+It provides a generic Python project structure
+and allows developers to keep tools and techniques in sync between multiple
+Python projects. It integrates the following tools:
 
 - pip and setuptools_scm for version management
 - Pre-commit with black, flake8 and isort for static analysis
@@ -26,20 +34,17 @@ It integrates the following tools:
   - which verifies all the things that CI does
 - If you use VSCode, it will run black, flake8, isort and mypy on save
 
-The related skeleton_ repo for this module contains the source
-code that can be merged into new or existing projects, and pulled from to
-keep them up to date. It can also serve as a working example for those who
-would prefer to cherry-pick.
 
-.. _skeleton: https://github.com/DiamondLightSource/python3-pip-skeleton
+.. _python3-pip-skeleton: https://github.com/DiamondLightSource/python3-pip-skeleton
+
+Quick start
+-----------
 
-This ``python3-pip-skeleton-cli`` repo contains the
-docs and a command line tool to ease the adoption of this skeleton into a
-new project like this::
+To create a new project based on skeleton::
 
     python3-pip-skeleton new /path/to/be/created --org my_github_user_or_org
 
-and existing projects::
+or to adopt skeleton into existing projects::
 
     python3-pip-skeleton existing /path/to/existing/repo --org my_github_user_or_org
 

docs/user/explanations/skeleton.rst docs/developer/explanations/skeleton.rstdocs/user/explanations/skeleton.rst renamed to docs/developer/explanations/skeleton.rst

@@ -0,0 +1,74 @@
+Pinning Requirements
+====================
+
+Introduction
+------------
+
+By design this project only defines dependencies in one place, i.e. in
+the ``requires`` table in ``pyproject.toml``.
+
+In the ``requires`` table it is possible to pin versions of some dependencies
+as needed. For library projects it is best to leave pinning to a minimum so
+that your library can be used by the widest range of applications.
+
+When CI builds the project it will use the latest compatible set of
+dependencies available (after applying your pins and any dependencies' pins).
+
+This approach means that there is a possibility that a future build may
+break because an updated release of a dependency has made a breaking change.
+
+The correct way to fix such an issue is to work out the minimum pinning in
+``requires`` that will resolve the problem. However this can be quite hard to
+do and may be time consuming when simply trying to release a minor update.
+
+For this reason we provide a mechanism for locking all dependencies to
+the same version as a previous successful release. This is a quick fix that
+should guarantee a successful CI build.
+
+Finding the lock files
+----------------------
+
+Every release of the project will have a set of requirements files published
+as release assets.
+
+For example take a look at the release page for python3-pip-skeleton-cli here:
+https://github.com/DiamondLightSource/python3-pip-skeleton-cli/releases/tag/3.3.0
+
+There is a list of requirements*.txt files showing as assets on the release.
+
+There is one file for each time the CI installed the project into a virtual
+environment. There are multiple of these as the CI creates a number of
+different environments.
+
+The files are created using ``pip freeze`` and will contain a full list
+of the dependencies and sub-dependencies with pinned versions.
+
+You can download any of these files by clicking on them. It is best to use
+the one that ran with the lowest Python version as this is more likely to
+be compatible with all the versions of Python in the test matrix.
+i.e. ``requirements-test-ubuntu-latest-3.8.txt`` in this example.
+
+Applying the lock file
+----------------------
+
+To apply a lockfile:
+
+- copy the requirements file you have downloaded to the root of your
+  repository
+- rename it to requirements.txt
+- commit it into the repo
+- push the changes
+
+The CI looks for a requirements.txt in the root and will pass it to pip
+when installing each of the test environments. pip will then install exactly
+the same set of packages as the previous release.
+
+Removing dependency locking from CI
+-----------------------------------
+
+Once the reasons for locking the build have been resolved it is a good idea
+to go back to an unlocked build. This is because you get an early indication
+of any incoming problems.
+
+To restore unlocked builds in CI simply remove requirements.txt from the root
+of the repo and push.

docs/developer/how-to/pin-requirements.rst

@@ -1,3 +1,5 @@
+.. _using pytest:
+
 Run the tests using pytest
 ==========================
 

docs/developer/how-to/run-tests.rst

@@ -0,0 +1,25 @@
+Container Local Build and Test
+==============================
+
+CI builds a runtime container for the project. The local tests
+checks available via ``tox -p`` do not verify this because not
+all developers will have docker installed locally.
+
+If CI is failing to build the container, then it is best to fix and
+test the problem locally. This would require that you have docker
+or podman installed on your local workstation.
+
+In the following examples the command ``docker`` is interchangeable with
+``podman`` depending on which container cli you have installed.
+
+To build the container and call it ``test``::
+
+    cd <root of the project>
+    docker build -t test .
+
+To verify that the container runs::
+
+    docker run -it test --help
+
+You can pass any other command line parameters to your application
+instead of --help.

docs/developer/how-to/test-container.rst

@@ -32,6 +32,8 @@ side-bar.
             how-to/lint
             how-to/update-tools
             how-to/make-release
+            how-to/pin-requirements
+            how-to/test-container
 
         +++
 
@@ -43,7 +45,7 @@ side-bar.
             :caption: Explanations
             :maxdepth: 1
 
-            explanations/decisions
+            explanations/skeleton
 
         +++
 

docs/developer/index.rst

@@ -1,5 +1,24 @@
-About the documentation
------------------------
+.. _documentation structure:
+
+3. Standard documentation structure
+===================================
+
+Date: 2023-01-18
+
+Status
+------
+
+Accepted
+
+Context
+-------
+
+Skeleton based project's documentation requires organizing.
+
+Decision
+--------
+
+Use the approach proposed by David Laing.
 
   :material-regular:`format_quote;2em`
 
@@ -16,3 +35,8 @@ approaches to their creation. Understanding the implications of this will help
 improve most documentation - often immensely.
 
 `More information on this topic. <https://documentation.divio.com>`_
+
+Consequences
+------------
+
+See article linked above.

docs/developer/explanations/decisions.rst docs/user/explanations/decisions.rstdocs/developer/explanations/decisions.rst renamed to docs/user/explanations/decisions.rst

@@ -1,8 +1,25 @@
-Why use a source directory
-==========================
+.. _src:
 
-This skeleton repo has made the decision to use a source directory. The reasons
-for this are set out in `Hynek's article`_ and summarized below.
+4. Use a source directory
+=========================
+
+Date: 2023-01-18
+
+Status
+------
+
+Accepted
+
+Context
+-------
+
+We need to decide how to structure the source code in skeleton based projects.
+
+
+Decision
+--------
+
+As per `Hynek's article`_ and summarized below.
 
 .. _Hynek's article: https://hynek.me/articles/testing-packaging/
 
@@ -26,3 +43,8 @@ This is tested in CI in the following way:
   checks that all files needed for the tests are packaged with the distribution.
 
 .. _editable install: https://pip.pypa.io/en/stable/cli/pip_install/#editable-installs
+
+Consequences
+------------
+
+See the article linked above.