changes after review01 with Tom

Author: gilesknap

.vscode/settings.json

@@ -15,6 +15,7 @@
     "cSpell.words": [
         "Beamlines",
         "blxxi",
-        "blxxt"
+        "blxxt",
+        "klogout"
     ]
 }

docs/developer/tutorials/dev-install.rst

@@ -42,7 +42,7 @@ requires python 3.8 or later) or to run in a container under `VSCode
 
         .. code::
 
-            $ vscode epics-containers.github.io
+            $ code epics-containers.github.io
             # Click on 'Reopen in Container' when prompted
             # Open a new terminal
 

docs/user/explanations/k8s-resources.rst

@@ -22,7 +22,6 @@ side-bar.
             tutorials/05a_ioc_changes
             tutorials/06_generic_ioc
             tutorials/07_support_module
-            tutorials/08_new_support
 
         +++
 
@@ -59,7 +58,6 @@ side-bar.
             explanations/docs-structure
             explanations/repositories
             explanations/cli-tools
-            explanations/k8s-resources
 
         +++
 
@@ -73,6 +71,8 @@ side-bar.
 
             reference/faq
             reference/cli
+            reference/ioc_helm_chart
+            reference/k8s_resources
             ../genindex
 
         +++

docs/user/index.rst

@@ -0,0 +1,70 @@
+IOC Helm Chart Details
+======================
+
+IOC instances are described using a helm chart which in turn generates
+a Kubernetes manifest to tell Kubernetes what resources to create in order
+to run for the IOC instance.
+
+Here we will look inside the template IOC instance in the template
+domain repository blxxi-template.
+
+You can see these files in Github here:
+https://github.com/epics-containers/blxxi-template/tree/main/iocs/blxxi-ea-ioc-01
+
+
+Examine the Example IOC Instance
+--------------------------------
+
+Take a look in the folder iocs/blxxi-ea-ioc-01. This contains a helm chart
+that defines the example IOC instance. To make new IOC instance you could
+copy this folder into your own domain repository's iocs folder. You would then
+need to rename it and make a few changes. Below is a description of the files
+in this folder and what you would need to change in your new IOC instance:
+
+-   ``Chart.yaml`` - this is the helm chart definition file. It contains
+    metadata about the chart and a list of dependencies. Most of this file
+    can be left as is for all IOC instances. But you do need to change these
+    fields:
+
+    - ``name`` - the unique name for the chart and the IOC instance it represents.
+    - ``Description`` - a short description of the IOC instance.
+
+-   ``values.yaml`` - this is the helm values file. It contains the values that
+    are substituted in to the helm templates when the helm chart is built. Most
+    of the values that go into an IOC instance chart will be drawn from
+    domain defaults which can be found in the folder ``beamline-chart``. Values
+    you need to supply here are:
+
+    -   ``base_image`` - the generic IOC image to use for this IOC instance. A
+        generic IOC image contains all the necessary support modules for a
+        given class of device and a compiled IOC binary with all those modules
+        linked. The IOC instance we are defining in a helm chart provides the startup
+        script and possibly database that makes this IOC instance unique. In this
+        case the generic IOC is for the area-detector simulator device.
+
+    -   ``prefix`` - the EPICS PV prefix for this IOC instance. This will set an
+        environment variable IOC_PREFIX which declares the prefix for the IOC's
+        devIOCStats records. You can leave this value out if you use the ioc
+        name as the prefix, but in this case we have used an uppercase version of
+        IOC name as the prefix.
+
+-   ``templates/ioc.yaml`` this is the master template for this helm chart,
+    it pulls in all the other templates from our dependencies. This file
+    has to appear here but is boilerplate and should not need to be changed.
+
+-   ``config`` this folder contains any files unique to this IOC instance. At
+    runtime on the cluster when the generic IOC image is running it will see
+    these files as mounted into the folder ``/repos/epics/ioc/config``.
+    In this case we have an EPICS startup script ``st.cmd`` only
+    and the default behaviour is just to run the IOC binary and pass it
+    ``st.cmd``.
+
+    To see how the Generic IOC makes use of the config folder take a look
+    at `this bash script`_ which runs on Generic IOC startup.
+
+    If you want to have completely custom behaviour in your IOC instance,
+    you can place a bash script called ``start.sh`` in the config folder
+    this will run in place of the above script.
+
+
+.. _this bash script:  https://github.com/epics-containers/ioc-template/blob/main/ioc/start.sh

docs/user/reference/ioc_helm_chart.rst

@@ -0,0 +1,40 @@
+Kubernetes Resources in an IOC Instance
+=======================================
+
+Learning about Helm and Kubernetes Manifests
+--------------------------------------------
+
+It is instructive to see what helm is doing when you deploy an IOC.
+
+Helm uses templates to generate a Kubernetes Manifest which defines a set
+of resources. It applies this manifest to the cluster using kubectl.
+
+To inspect the kubernetes manifest YAML that is created when we deploy the
+example IOC you can use the following command from inside of your example
+beamline repository:
+
+.. code-block:: bash
+
+    ec ioc template iocs/bl01t-ea-ioc-01
+
+This is expanding the local helm chart in the ``iocs/bl01t-ea-ioc-01`` folder and using
+its ``templates/ioc.yaml`` plus the templates in ``helm-ioc-lib``. These templates
+are expanded using the values in the ``iocs/bl01t-ea-ioc-01/values.yaml`` file and also
+``beamline-chart/values.yaml`` and finally the default ``values.yaml`` file
+from the helm-ioc-lib.
+
+For a description of the key resources we create in this Kubernetes manifest
+see the next heading.
+
+
+Kubernetes Resources in an IOC Instance
+---------------------------------------
+
+TODO - This is a work in progress.
+
+The key resources that we are asking Kubernetes to create are:
+
+- ``Deployment``. Tells Kubernetes to run one and only one generic IOC
+  container. See deployments:
+
+  https://kubernetes.io/docs/concepts/workloads/controllers/deployment/

docs/user/reference/k8s_resources.rst

@@ -19,7 +19,7 @@ EPICS, WSL2 and more.
 .. Note::
 
     **DLS Users**: RHEL 8 Workstations at DLS have podman installed by default.
-    You can access vscode with ``module load vscode``. RHEL 7 Workstations
+    You can access VSCode with ``module load vscode``. RHEL 7 Workstations
     are not supported.
 
 Options

docs/user/tutorials/01_setup_workstation.rst

@@ -16,8 +16,8 @@ container definition is in https://github.com/epics-containers/.devcontainer.
 Configure Visual Studio Code
 ----------------------------
 
-For podman users, you must first tell vscode to use podman instead of docker.
-Open a vscode window and hit "ctrl ," (control-comma) to open the user
+For podman users, you must first tell VSCode to use podman instead of docker.
+Open a VSCode window and hit "ctrl ," (control-comma) to open the user
 settings editor and search for
 "dev.containers.dockerPath", change its value from "docker" to "podman".
 
@@ -49,10 +49,10 @@ to reopen the folder in a container. Click on the ``Reopen in Container`` button
 
     reopen in container dialogue
 
-Now all of your vscode terminals and file explorer will be running inside of
+Now all of your VSCode terminals and file explorer will be running inside of
 the devcontainer and have access to all the tools installed there.
 
-To verify things are working as expected, open a terminal in vscode from
+To verify things are working as expected, open a terminal in VSCode from
 the menus ``Terminal > New Terminal``. You should see a prompt like this:
 
 .. code-block:: bash
@@ -77,7 +77,8 @@ Things to note:
   to ``/root``.
 
 - Podman users are running as root inside the container but files will be
-  written with your user id and group id.
+  written with your user id and group id when writing to the mounted
+  workspace folder (or your home directory).
 
 .. _devcontainer-configure:
 
@@ -90,79 +91,56 @@ Configuring the Devcontainer
     configured for interacting with the test beamline bl01t on the test
     cluster Pollux. HOWEVER: for this exercise we will use your personal
     GitHub account to avoid clashes with other users of this tutorial.
-    Therefore follow the instructions below and for the KUBECONFIG setting
-    use the following:
+    Therefore follow the instructions below and set KUBECONFIG as follows:
 
     .. code-block:: bash
 
         # point at your cluster configuration file
         export KUBECONFIG=/home/${USER}/.kube/config_pollux
 
     To enable access to the pollux cluster, execute the following commands
-    from outside of the dev container:
+    from OUTSIDE of the dev container:
 
     .. code-block:: bash
 
         module load pollux
+        klogout
         .devcontainer/dls-copy-k8s-crt.sh # a script in the .devcontainer repo
         kubectl get nodes
 
-    The last command will ask for your fed-id and password and then show A
-    list of nodes in the pollux cluster.
+    The last command will ask for your fed-id and password and then show a
+    list of nodes in the pollux cluster. Your credentials are cached for a
+    week after which you will see authentication errors. To fix this
+    repeat the above steps.
 
 You devcontainer environment is configured by a file called
 ``.bashrc_dev`` file. The terminals in the devcontainer will source this
 file when they start.
 
-You can take a copy of ``.devcontainer/.bashrc_dev`` and place it in your
-home folder to customize it.
-i.e.:
-
-
-    # IMPORTANT: use /home/$USER not $HOME
-    cp .devcontainer/.bashrc_dev /home/${USER}/.bashrc_dev
-    code /home/${USER}/.bashrc_dev
-
-Alternatively you can take a fork of the .devcontainer repo and make your
-own version of the .bashrc_dev file in place.
-
-Much of this file is setting up convenience features like prompt and bash
+Much of this file is setting up convenience features like the prompt and bash
 history. You can change these to suit your own preferences.
 
 The primary configuration options are the environment variables exported by
 this script. These are listed below with some recommended values for running
 these tutorials. Paste the following into the ``.bashrc_dev`` file and
-change GITHUB_ORG to your GitHub organization or user.
+add your GitHub organization or user to K8S_HELM_REGISTRY.
 
 .. code-block:: bash
 
     ############ REPLACE all environment below with your details ###################
 
-    # Github organization or user name
-    export GITHUB_ORG=<YOUR GITHUB ORGANIZATION OR USER GOES HERE>
-
-    # point at your cluster configuration file
+    # point at your cluster config file
     export KUBECONFIG=/home/${USER}/.kube/config
 
-    # the default beamline or domain for ec commands
-    export BEAMLINE=t01 # equivalent to K8S_DOMAIN=bl01t
+    # the default domain for ec commands (REMOVE if this is supplied by the host)
+    export K8S_DOMAIN=bl01t
 
     # where to get HELM charts for ec commands
-    export K8S_HELM_REGISTRY=ghcr.io/${GITHUB_ORG}
+    export K8S_HELM_REGISTRY=ghcr.io/<YOUR GITHUB USER OR ORGANIZATION>
 
-    # where to get container IMAGES for ec commands
-    export K8S_IMAGE_REGISTRY=ghcr.io/${GITHUB_ORG}
+    ################################################################################
 
-    # the URL for the facility logging system
-    export K8S_LOG_URL='none'
-
-    # set this to True to suppress output of commands in 'ec' CLI
-    unset K8S_QUIET
-
-    # extra arguments to supply to containerized CLI commands
-    export K8S_CLI_ARGS=''
-
-After editing ``/home/$USER/.bashrc_dev`` you will need to close any open terminals and
+After editing ``.bashrc_dev`` you will need to close any open terminals and
 restart them to pick up the changes.
 
 
@@ -177,6 +155,9 @@ restart them to pick up the changes.
     In addition, you can alter the system packages installed in the container or make
     any other changes to the Dockerfile and regenerate your own container image.
 
+    To pick up such changes to ``.devcontainer`` run the ``Rebuild Container``
+    command from VSCode command pallette (accessed via ctrl-shift-P).
+
     If you wish to persist these changes
     then it is suggested that you make your own github repo of .devcontainer and
     push the changes there.

docs/user/tutorials/02_devcontainer.rst

@@ -3,6 +3,22 @@
 Setup a Kubernetes Server
 =========================
 
+.. note::
+
+    From this point onward the tutorials assume that you are using
+    Kubernetes and Helm to deploy your EPICS IOCs.
+
+    However:
+
+    The approach of creating generic IOC's in containers and then deploying
+    IOC instances as generic IOC's + some configuration will also work
+    standalone, or with other orchestration tools. e.g. In `05a_ioc_changes`
+    we will demonstrate running an IOC locally using podman alone.
+
+    If you want to take advantage of containers for IOCs but do not want to
+    take on Kubernetes then there are some hints as to how to achieve this
+    here: `../how-to/run_iocs`.
+
 Introduction
 ------------
 This is a very easy set of instructions for setting up an experimental
@@ -12,7 +28,6 @@ ready to test deployment of EPICS IOCs.
 Bring Your Own Cluster
 ----------------------
 
-
 If you already have a Kubernetes cluster then you can skip this section.
 and go straight to `./04_create_beamline`.
 

docs/user/tutorials/03_setup_k8s.rst

@@ -90,7 +90,7 @@ repo in the future.
         sed -i 's/BLXXI/BL01T/g' $(find * -type f)
         sed -i 's/blxxi/bl01t/g' $(find * -type f)
 
-#.  Add your new repo to your vscode workspace and take a look at what you
+#.  Add your new repo to your VSCode workspace and take a look at what you
     have.
 
     From the VSCode menus: File->Add Folder to Workspace