fixup essential features

Author: gilesknap

docs/conf.py

@@ -8,9 +8,8 @@
 from pathlib import Path
 from subprocess import check_output
 
-import requests
-
 import epics_containers
+import requests
 
 # -- General configuration ------------------------------------------------
 

docs/developer/how-to/update-tools.rst

@@ -1,7 +1,7 @@
 Update the tools
 ================
 
-This module is merged with the python3-pip-skeleton_. This is a generic
+This module is merged with the python3-pip-skeleton_. This is a Generic
 Python project structure which provides a means to keep tools and
 techniques in sync between multiple Python projects. To update to the
 latest version of the skeleton, run::

docs/user/explanations/introduction.rst

@@ -16,7 +16,7 @@ Concepts
 Images and Containers
 ~~~~~~~~~~~~~~~~~~~~~
 Containers provide the means to package up IOC software and execute it
-in a lightweight virtual environment. This includes saving the packages
+in a lightweight virtual environment. These packages are then saved
 into public or private image registries such as DockerHub or Github Container
 Registry.
 
@@ -37,19 +37,35 @@ code to suit your infrastructure. At DLS, this means that we are able to use
 vanilla EPICS base and support modules. We no longer require our own
 forks of these repositories.
 
+.. _generic iocs:
+
 Generic IOCs and instances
 """"""""""""""""""""""""""
 
 An important principal of the approach presented here is that an IOC container
-image represents a 'generic' IOC. The generic IOC image is used for all
-IOC instances that connect to a given class of device.
+image represents a 'Generic' IOC. The Generic IOC image is used for all
+IOC instances that connect to a given class of device. For example the
+Generic IOC image here:
+ghcr.io/epics-containers/ioc-adaravis-linux-runtime:2023.10.1
+uses the AreaDetector driver ADAravis to connect to GigE cameras.
+
+An IOC instance runs in a container runtime by loading two things:
+
+- The Generic IOC image passed to the container runtime.
+- The IOC instance configuration. This is mapped into the container at
+  runtime by mounting it into the filesystem at runtime. The mount point
+  for this configuration is alway /epics/ioc/config.
+
+The configuration will bootstrap the unique properties of that instance.
+The following contents for the configuration are supported:
+
+- ioc.yaml: an **ibek** IOC description file which **ibek** will use to generate
+  st.cmd and ioc.subst
+- st.cmd, ioc.subst: an IOC shell startup script and an optional substitution file
+  st.cmd can refer any additional files in the configuration directory
+- start.sh a bash script to fully override the startup of the IOC. start.sh
+  can refer to any additional files in the configuration directory
 
-An IOC instance runs in a container that bases its
-filesystem on a generic IOC image.
-In addition the instance has configuration mapped into the
-container that will bootstrap the unique properties of that instance.
-In most cases the configuration need only be a single IOC boot script or
-IBEK yaml file.
 
 This approach reduces the number of images required and saves disk. It also
 makes for simple configuration management.
@@ -70,23 +86,33 @@ since it was open-sourced in 2014.
 
 Today it is by far the dominant orchestration technology for containers.
 
-In this project we use Kubernetes to provide a standard way of implementing
-these features:
+In this project we use Kubernetes and helm to provide a standard way of
+implementing these features:
 
 - Auto start IOCs when servers come up
 - Manually Start and Stop IOCs
 - Monitor IOC status and versions
 - Deploy versioned IOCs to the beamline
 - Rollback to a previous IOC version
-- Allocate the server which runs an IOC
+- Allocate a server with adequate resources on which to run each IOC
 - Failover to another server (for soft IOCs not tied to hardware in a server)
 - View the current log
-- View historical logs (via graylog)
+- View historical logs (via graylog or other centralized logging system)
 - Connect to an IOC and interact with its shell
 - debug an ioc by starting a bash shell inside it's container
-- etc.
 
 
+Kubernetes Alternative
+~~~~~~~~~~~~~~~~~~~~~~
+
+If you do not have the resources to maintain a Kubernetes cluster then this project
+is experimentally supporing the use of podman-compose or docker-compose to deploy
+IOCs to a single server. Where a beamline has multiple servers the distribution of
+IOCs across those servers is managed by the user. These tools would replace
+Kubernetes and Helm in the technology stack.
+
+TODO: more on this once we have a working example.
+
 Helm
 ~~~~
 https://helm.sh/
@@ -102,16 +128,27 @@ of the chart within the cluster.
 It also supports registries for storing version history of charts,
 much like docker.
 
-In this project we use Helm Charts to define and deploy IOC instances. Each
-beamline has its own Helm Repository which stores current and historical
-version of its IOC instances. Each IOC has a Helm Chart which defines the
-which generic IOC image it is based on and the configuration that makes it into
-an individual IOC instance.
+In this project we use Helm Charts to define and deploy IOC instances.
+Each beamline (or accelerator domain) has its own git repository that holds
+the beamline Helm Chart for its IOCs. Each IOC instance need only provide a
+values.yaml file to override the default values in the Helm Chart and a config folder
+as described in `generic iocs`.
+
+**epics-containers** does not use helm repositories for storing IOC instances.
+Such repositories only hold a zipped version of the chart and a values.yaml file,
+and this is seen as redundant when we have a git repository holding the same
+information. Instead we provide a command line tool for installing and updating
+IOCs. Which performs the following steps:
+
+- Clone the beamline repository at a specific tag to a temporary folder
+- extract the beamline chart and apply the values.yaml to it
+- additionally generate a config map from the config folder files
+- install the resulting chart into the cluster
+- remove the temporary folder
+
+This means that we don't store the chart itself but we do store all of the
+information required to re-generate it in a version tagged repository.
 
-With the epics-containers approach there is a 1:1 relationship between
-Helm Charts and IOC instances. These helm charts therefore hold all the
-information about an IOC instance and can be deployed with no additional
-configuration parameters.
 
 Repositories
 ~~~~~~~~~~~~
@@ -125,56 +162,51 @@ a shared filesystem are required
 (The legacy approach at DLS relied heavily on
 know locations in a shared filesystem).
 
-In the epics-containers examples all repositories are held in the same
+In the **epics-containers** examples all repositories are held in the same
 github organization. This is nicely contained and means that only one set
 of credentials is required to access all the resources.
 
 There are many alternative services for storing these repositories, both
 in the cloud and on premises. Below we list the choices we have tested
 during the POC.
 
-The 3 classes of repository are as follows:
+The 2 classes of repository are as follows:
 
 :Source Repository:
+
   - Holds the source code but also provides the
     Continuous Integration actions for testing, building and publishing to
     the image / helm repositories. These have been tested:
 
     - github
     - gitlab (on premises)
 
-    epics-containers defines two classes of source repository:
+  - epics-containers defines two classes of source repository:
 
-    - Generic IOC source. Defines how a generic IOC image is built, this does
+    - Generic IOC source. Defines how a Generic IOC image is built, this does
       not typically include source code, but instead is a set of instructions
-      for building the generic IOC image.
+      for building the Generic IOC image.
     - Beamline / Accelerator Domain source. Defines the IOC instances for a
       beamline or Accelerator Domain. This includes the IOC boot scripts and
-      any other configuration required to make the IOC instance unique, for
-      IBEK based IOCs, each IOC instance is defined by an IBEK yaml file only.
+      any other configuration required to make the IOC instance unique.
+      For **ibek** based IOCs, each IOC instance is defined by an **ibek**
+      yaml file only.
 
 :An OCI image repository:
-  - Holds the generic IOC container images and their
+
+  - Holds the Generic IOC container images and their
     dependencies. The following have been tested:
 
     - Github Container Registry
     - DockerHub
     - Google Cloud Container Registry
 
-:An OCI helm chart repository:
-  - This is where the definitions of IOC instances
-    are stored. They are in the form of a helm chart which describes to
-    Kubernetes the resources needed to spin up the IOC.
-    These have been tested:
-
-    - github packages
-    - Google Cloud Artifact Registry
 
 Continuous Integration
 ~~~~~~~~~~~~~~~~~~~~~~
 
 Our examples all use continuous integration to get from pushed source
-to the published images / helm charts.
+to the published images.
 
 This allows us to maintain a clean code base that is continually tested for
 integrity and also to maintain a direct relationship between source code tags
@@ -183,19 +215,17 @@ and the tags of their built resources.
 There are these types of CI:
 
 :Generic IOC source:
-    - builds a generic IOC container image
+    - builds a Generic IOC container image
     - runs some tests against that image
     - publishes the image to github packages (only if the commit is tagged)
+      or other OCI registry
 
 :beamline definition source:
     - builds a helm chart from each ioc definition
-    - IF the commit is tagged then those IOCs that have changed since the last
-      tag will be published to the helm OCI repository with the same version
-      tag.
-
-:helm library source:
-    - builds the helm library chart shared by all Generic IOCs.
-    - publishes it to github packages (only if the commit is tagged)
+    - tests that the helm chart is deployable (but does not deploy it)
+    - locally launches each IOC instance and loads its configuration to
+      verify that the configuration is valid (no system tests because this
+      would require talking to beamline hardware).
 
 :documentation source:
     - builds the sphinx docs
@@ -208,34 +238,48 @@ on MVME5500 hardware. Soft IOCs that require access to hardware on the
 server (e.g. USB or PCIe) will be supported by mounting the hardware into
 the container (theses IOCS will not support failover).
 
-Other linux architectures could be added to the Kubernetes cluster.
+Other linux architectures could be added to the Kubernetes cluster. We have
+tested arm64 native builds and will add this as a supported architecture
+in the future.
 
-Note that OPI is also out of scope for this initial phase. See
-`no_opi`
 
 Additional Tools
 ----------------
 
 epics-containers-cli
 ~~~~~~~~~~~~~~~~~~~~
-The project epics-containers-cli implements simple command
-line functions for deploying and monitoring IOCs. It is just a wrapper
-around the tools kubectl, podman and helm, but saves typing and provides
-help and command line completion.
+This define the developer's 'outside of the container' helper tool. The command
+line entry point is **ec**.
+
+The project is a python package featuring simple command
+line functions for deploying, monitoring building and debugging
+Generic IOCs and IOC instances. It is a wrapper
+around the standard command line tools kubectl, podman/docker, helm, and git
+but saves typing and provides help and command line completion.
 
 See `CLI` for details.
 
-dev-e7
-~~~~~~
-This project provides a Dockerfile for building a personal developer container,
-allowing a developer to work on support modules or IOCs anywhere. All the
-tools required to build and deploy IOCs are included in the container.
 
+**ibek**
+~~~~~~~~
+IOC Builder for EPICS and Kubernetes is the developer's 'inside the container'
+helper tool. It is a python package that is installed into the Generic IOC
+container images. It is used:
 
-Ibek
-~~~~
-IOC Builder for EPICS and Kubernetes provides a way to generate an IOC
-helm chart from a YAML description of the IOC.
+- at container build time to fetch and build EPICS support modules
+- to generate the IOC source code and compile it
+- to extract all useful build artifacts into a runtime image
 
 See https://github.com/epics-containers/ibek.
 
+PVI
+~~~
+Process Variables Interface is a python package that is installed into the
+Generic IOC container images. It is used to give structure to the IOC's PVI
+interface allowing us to:
+
+- add metadata to the IOCs DB records for use by bluesky
+- auto generate screens for the device (as bob, adl or edm files)
+
+
+

docs/user/explanations/kubernetes_cluster.rst

@@ -91,14 +91,14 @@ Argus is a multi-tenant cluster. Namespaces are used to enforce multi-tenancy. A
 Beamline Local Cluster Nodes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-As part of the investigation work some worker nodes in Argus have been connected that are physically located at the beamline. These nodes do not share the same rack as Argus, and hence are part of a different routed subnet to the /22 that the control plane and main workers are within. This model assumes one centralised control plane (and some generic workers), and a set of beamline cluster nodes that may be distributed across the network (in different subnets).
+As part of the investigation work some worker nodes in Argus have been connected that are physically located at the beamline. These nodes do not share the same rack as Argus, and hence are part of a different routed subnet to the /22 that the control plane and main workers are within. This model assumes one centralised control plane (and some Generic workers), and a set of beamline cluster nodes that may be distributed across the network (in different subnets).
 
 The beamline cluster nodes require a few interesting sets of configuration to make this architecture work. See the following subheadings for details.
 
 Metallb Pools
 +++++++++++++
 
-Metallb cannot be used to provide loadBalancer services for pods running on the beamline cluster nodes. This is because metallb currently only supports a single pool of IPs to allocate from. In the case of Argus, the pool is allocated from within the /22 in which the control plane (and a few generic workers) sit. Should a pod with a loadBalancer Service IP get brought up on a beamline cluster node, the traffic would not be routable because the beamline TOR switch does not send ARP messages for subnets that it is not directly connected to. This is not an issue running IOCs since they do not make use of loadBalancer Services. There is a feature request for Metallb to support address pools that is currently pending.
+Metallb cannot be used to provide loadBalancer services for pods running on the beamline cluster nodes. This is because metallb currently only supports a single pool of IPs to allocate from. In the case of Argus, the pool is allocated from within the /22 in which the control plane (and a few Generic workers) sit. Should a pod with a loadBalancer Service IP get brought up on a beamline cluster node, the traffic would not be routable because the beamline TOR switch does not send ARP messages for subnets that it is not directly connected to. This is not an issue running IOCs since they do not make use of loadBalancer Services. There is a feature request for Metallb to support address pools that is currently pending.
 
 Node Labelling and Taints
 +++++++++++++++++++++++++

docs/user/explanations/repositories.rst

@@ -5,10 +5,10 @@ Source and Registry Locations
 
     **DLS Users** DLS is currently using these locations for assets:
 
-    - Generic IOC Source: ``https://github.com/epics-containers<generic ioc>``
+    - Generic IOC Source: ``https://github.com/epics-containers<Generic ioc>``
     - Beamline Source repos: ``https://gitlab.diamond.ac.uk/controls/containers/beamline/<beamline>``
     - Accelerator Source repos: ``https://gitlab.diamond.ac.uk/controls/containers/accelerator/<domain>``
-    - Generic IOC Container Images: ``ghcr.io/epics-containers/<generic ioc>``
+    - Generic IOC Container Images: ``ghcr.io/epics-containers/<Generic ioc>``
     - IOC Instance Helm Charts: ``helm-test.diamond.ac.uk/iocs/<domain>/<ioc instance>``
 
 Where to Keep Source Code
@@ -28,11 +28,11 @@ also contribute to the development of the IOC.
 
 The intention is that a Generic IOC container image is a reusable component
 that can be used by multiple IOC instances in multiple domains. Because
-generic IOCs are containerized and not facility specific, they should work
+Generic IOCs are containerized and not facility specific, they should work
 anywhere. Therefore these make sense as public repositories.
 
 There may be cases where the this is not possible, for example if the
-generic IOC relies on proprietary support modules with restricted licensing.
+Generic IOC relies on proprietary support modules with restricted licensing.
 
 The existing Continuous
 Integration files for Generic IOCs work with GitHub actions, but also

docs/user/how-to/run_iocs.rst

@@ -1,9 +1,9 @@
 Run an IOC without Kubernetes
 =============================
 
-When we run a containerized IOC we are selecting a generic IOC container image
+When we run a containerized IOC we are selecting a Generic IOC container image
 to launch and some configuration to pass to the IOC.  The configuration we
-pass is what makes the IOC a unique instance instead of a generic IOC.
+pass is what makes the IOC a unique instance instead of a Generic IOC.
 
 In the tutorials we are running the IOC in a Kubernetes cluster and we
 use a Kubernetes ConfigMap to pass the configuration to the IOC.
@@ -19,7 +19,7 @@ will see the IOC running in a container:
 
     podman run --net host -it --rm -v $(pwd)/iocs/bl01t-ea-ioc-01/config:/repos/epics/ioc/config  ghcr.io/epics-containers/ioc-template-linux-runtime:23.3.3
 
-What you have done here is pull down the generic ``ioc-template-linux-runtime``
+What you have done here is pull down the Generic ``ioc-template-linux-runtime``
 container image from the GitHub Container Registry and run it.
 The ``-it`` means it is an interactive terminal session.  ``--rm`` means the
 container is removed when it exits, best for keeping your cache tidy.
@@ -37,7 +37,7 @@ is appended to the path and that is required because the path for mounting
 into a container must be absolute.
 
 ``iocs/bl01t-ea-ioc-01/config`` contains a simple iocShell boot script and the
-default behaviour is to pass that to the generic IOC binary.
+default behaviour is to pass that to the Generic IOC binary.
 
 Using this approach you could manage your config folders and IOC launches
 yourself using whatever mechanism you prefer. Probably docker compose or

docs/user/how-to/update_templated.rst

@@ -10,7 +10,7 @@ The tutorials use two template projects as follows:
     * - Template Project
       - Purpose
     * - ioc-template
-      - create a new generic IOC container project
+      - create a new Generic IOC container project
     * - blxxi-template
       - create a new IOC Instance Domain project
 

docs/user/overview.rst

@@ -1,5 +1,5 @@
-Kubernetes for EPICS IOCs applies modern industry standards to the management
-of IOCs.
+**epics-containers** applies modern industry best practice for software
+delivery to the management of EPICS IOCs.
 
 There are 5 themes to this strategy: