update tutorials up to dev_container with Jan 2024 framework changes

Author: gilesknap

docs/conf.py

@@ -15,7 +15,7 @@
 
 # General information about the project.
 project = "epics-containers.github.io"
-copyright = "2023, Diamond Light Source"
+copyright = "2024, Diamond Light Source"
 author = "Giles Knap"
 
 # The full version, including alpha/beta/rc tags.

docs/user/how-to/phoebus.rst

@@ -9,4 +9,4 @@ This is the initial target for epics-containers GUIs, other OPI
 formats may be supported in the future.
 
 OPI file generation is work in progress and this page will be updated when
-it is ready (est Nov 2023).
+it is ready (est Feb 2024).

docs/user/tutorials/create_beamline.rst

@@ -110,10 +110,15 @@ Steps
     #. change the beamline name in the two bash scripts in the ``services``
        directory.
 
+    #. change the beamline name in ``beamline-chart/values.yaml``
+
     #. add some meaningful configuration to the example IOCs config folder
        ``iocs/bl01t-ea-ioc-01/config/ioc.yaml``. We will do this in the
        next tutorial.
 
+Much of the above can be achieved with a global find and replace of 'xxi' with
+'01t'.
+
 Environment.sh
 ~~~~~~~~~~~~~~
 
@@ -175,6 +180,14 @@ Open both files in ``services/`` and replace blxxi with bl01t.
 
 TODO: add support for local docker installations of these services.
 
+Change the Beamline Name in the Helm Chart
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The file ``beamline-chart/values.yaml`` contains the configuration values
+for the helm chart that is used to deploy the beamline. You can change
+the values ``blxxi`` to ``bl01t`` in here. But this will not be relevant until
+we get to the Kubernetes tutorial when the deployment is done using helm.
+
 Wrapping Up
 -----------
 
@@ -189,8 +202,8 @@ You can now push the repository up to GitHub and give it a version tag like this
     git add .
     git commit -m "changed blxxi to bl01t"
     git push
-    git tag 2023.11.1
-    git push origin 2023.11.1
+    git tag 2024.1.1
+    git push origin 2024.1.1
 
 
 We use ``CalVer`` version numbers for beamline repositories and Generic IOCs.
@@ -201,3 +214,9 @@ CalVer is described here: https://calver.org/ and is used where semantic
 versioning is not appropriate because the repository contains a mix of
 dependencies and does not have a clear API.
 
+Note that 2024.1.1 represents the time that this tutorial was last updated.
+For completeness you could use the current year and month instead. You
+are also free to choose your own versioning scheme as this is not enforced by
+any of the epics-containers tools.
+
+

docs/user/tutorials/create_ioc.rst

@@ -45,7 +45,7 @@ This should launch vscode and open the values.yaml file. Add the following:
 
 .. code-block:: yaml
 
-    image: ghcr.io/epics-containers/ioc-adsimdetector-linux-runtime:2023.11.1
+    image: ghcr.io/epics-containers/ioc-adsimdetector-linux-runtime:2024.1.1
 
 This tells the IOC Instance to run in the ``ioc-adsimdetector-linux-runtime``
 container. This container was built by the Generic IOC source repo here
@@ -71,7 +71,7 @@ To recap, we have two python CLI tools for supporting ``epics-containers``:
     - Generating startup scripts and EPICS databases at container runtime
       from *IOC yaml* files.
     - Fetching and compiling support modules at build time.
-    - Other build time utilities such as creating and compiling IOC source and
+    - Other build time utilities such as
       extracting runtime assets for the runtime container target.
 
 :ec: provides developer support *outside* of the container such as:
@@ -190,7 +190,7 @@ This should launch vscode and open the ioc.yaml file. Add the following:
 
 .. code:: yaml
 
-    # yaml-language-server: $schema=https://github.com/epics-containers/ioc-adsimdetector/releases/download/2023.11.1/ibek.ioc.schema.json
+    # yaml-language-server: $schema=https://github.com/epics-containers/ioc-adsimdetector/releases/download/2024.1.1/ibek.ioc.schema.json
 
     ioc_name: bl01t-ea-ioc-02
     description: Example simulated camera for BL01T
@@ -320,15 +320,15 @@ That is because every Generic IOC publishes an *IOC schema* that describes
 the set of entities that an instance of that IOC may instantiate.
 
 The Generic IOC we used was released at this location:
-https://github.com/epics-containers/ioc-adsimdetector/releases/tag/2023.11.1.
+https://github.com/epics-containers/ioc-adsimdetector/releases/tag/2024.1.1.
 This page includes the assets that are published as part of the release and
 one of those is ``ibek.ioc.schema.json``. This is the *IOC schema* for the
 ``ioc-adsimdetector`` Generic IOC. This is what we referred to at the top of
 our *IOC yaml* file like this:
 
 .. code:: yaml
 
-    # yaml-language-server: $schema=https://github.com/epics-containers/ioc-adsimdetector/releases/download/2023.11.1/ibek.ioc.schema.json
+    # yaml-language-server: $schema=https://github.com/epics-containers/ioc-adsimdetector/releases/download/2024.1.1/ibek.ioc.schema.json
 
 When editing with a YAML aware editor like VSCode this will enable auto
 completion and validation of the *IOC yaml* file. To enable this in VSCode

docs/user/tutorials/deploy_example.rst

@@ -52,7 +52,7 @@ The standard way to set up your environment for any domain is to get
 the environment.sh script from the domain repository and source it.
 
 First make sure you have the local binaries folder in your path by adding
-the following to the end of you $HOME/.bash_profile file:
+the following to the end of you ``$HOME/.bash_profile`` file:
 
 .. code-block:: bash
 
@@ -63,7 +63,7 @@ where indicated):
 
 .. code-block:: bash
 
-    cd /tmp
+    mkdir -p ~/.local/bin
     curl -o ~/.local/bin/bl01t https://raw.githubusercontent.com/**YOUR GITHUB ACCOUNT**/bl01t/main/environment.sh?token=$(date +%s)
     source ~/.bash_profile
     source bl01t
@@ -73,8 +73,8 @@ profile you should be able to enable the ``bl01t`` environment as follows:
 
 .. code-block:: bash
 
-    # first make sure you have loaded your virtual environment
-    source $HOME/ec-venv/bin/activate
+    # first make sure you have loaded your virtual environment for the ec tool
+    source $HOME/ec-venv/bin/activate # DLS users don't need this step
     source bl01t
 
 
@@ -117,18 +117,20 @@ You can now see the beta IOC instance running with:
 
     $ ec ps
     IOC NAME            VERSION             STATUS              IMAGE
-    bl01t-ea-ioc-01     2023.10.26-b11.53   Up 6 minutes        ghcr.io/epics-containers/ioc-adsimdetector-linux-runtime:2023.10.5
+    bl01t-ea-ioc-01     2024.1.19-b11.53   Up 6 minutes        ghcr.io/epics-containers/ioc-adsimdetector-linux-runtime:2024.1.1
 
 At the end of the last tutorial we tagged the beamline repository with a
 ``CalVer`` version number and pushed it up to GitHub. This means that we
-can now release the IOC instance with that same version number. First let's
-check that the IOC instance version is available as expected:
+can now use that tagged release of the IOC instance. First let's
+check that the IOC instance version is available as expected. The following
+command lists all of the tagged versions of the IOC instance that are
+available in the GitHub repository.
 
 .. code-block:: bash
 
     $ ec ioc instances bl01t-ea-ioc-01
     Available instance versions for bl01t-ea-ioc-01:
-        2023.11.1
+        2024.1.1
 
 .. note::
 
@@ -144,13 +146,27 @@ it to your local machine:
 
 .. code-block:: bash
 
-    $ ec ioc deploy bl01t-ea-ioc-01 2023.11.1
+    $ ec ioc deploy bl01t-ea-ioc-01 2024.1.1
     bdbd155d437361fe88bce0faa0ddd3cd225a9026287ac5e73545aeb4ab3a67e9
 
     $ ec ps
     IOC NAME            VERSION             STATUS              IMAGE
-    bl01t-ea-ioc-01     2023.11.1           Up 4 seconds        ghcr.io/epics-containers/ioc-adsimdetector-linux-runtime:2023.10.5
+    bl01t-ea-ioc-01     2024.1.1           Up 4 seconds        ghcr.io/epics-containers/ioc-adsimdetector-linux-runtime:2023.10.5
 
+IMPORTANT: deploy-local vs deploy
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Be aware of the distinction of ``deploy-local`` vs ``deploy``. Both of these
+commands create a running instance of the IOC in the target environment (currently
+your local machine - later on a Kubernetes Cluster). However, ``deploy-local``
+gets the IOC instance description YAML direct from your local filesystem. This
+means it is not likely to be available for re-deployment later on. ``deploy``
+gets the IOC instance description YAML from the GitHub repository with able
+specific tag and therefore is a known state that can be recovered at a later
+date.
+
+Always strive to have released versions of IOC instances deployed in your
+environments. ``deploy-local`` is only for temporary testing purposes.
 
 Managing the Example IOC Instance
 ---------------------------------
@@ -174,7 +190,7 @@ To stop / start the example IOC try the following commands. Note that
     Generic IOCs.
 
     You may have noticed that the IOC instance has is showing that it has
-    an image ``ghcr.io/epics-containers/ioc-adsimdetector-linux-runtime:2023.10.5``.
+    an image ``ghcr.io/epics-containers/ioc-adsimdetector-linux-runtime:2024.1.1``.
 
     This is a Generic IOC image and all IOC Instances must be based upon one
     of these images. This IOC instance has no startup script and is therefore
@@ -192,9 +208,11 @@ iocShell.
 
     ec ioc attach bl01t-ea-ioc-01
 
-Use the command sequence ctrl-P then ctrl-Q to detach from the IOC
-You can also usually restart and detach from the IOC using ctrl-D or
-ctrl-C.
+Use the command sequence ctrl-P then ctrl-Q to detach from the IOC. **However,
+there are issues with both VSCode and IOC shells capturing ctrl-P. until
+this is resolved it may be necessary to close the terminal window to detach.**
+You can also restart and detach from the IOC using ctrl-D or ctrl-C, or
+by typing ``exit``.
 
 To run a bash shell inside the IOC container:
 
@@ -205,11 +223,17 @@ To run a bash shell inside the IOC container:
 Once you have a shell inside the container you could inspect the following
 folders:
 
-=============== ==============================================================
-ioc code        /epics/ioc
-support modules /epics/support
-EPICS binaries  /epics/epics-base
-=============== ==============================================================
+===================    =======================================================
+ioc code               /epics/ioc
+support modules        /epics/support
+EPICS binaries         /epics/epics-base
+IOC instance config    /epics/ioc/config
+IOC startup script     /epics/runtime
+===================    =======================================================
+
+Being at a terminal prompt inside the IOC container can be useful for debugging
+and testing. You will have access to caget and caput, plus other EPICS tools,
+and you can can inspect files such as the IOC startup script.
 
 Logging
 ~~~~~~~

docs/user/tutorials/dev_container.rst

@@ -211,25 +211,38 @@ Preparing the IOC for Testing
     rm -rf ~/.vscode/* ~/.vscode-server/*
 
 Now that you are *inside* the container you have access to the tools built into
-it, this includes ``ibek``. The first command you should run is:
+it, this includes ``ibek``.
+
+The first commands you should run are as follows:
 
 .. code-block:: bash
 
-   ibek ioc build
-
-This generates an IOC source tree in the ``ioc`` folder under your
-``ioc-adsimdetector`` folder and compiles it. Note that the IOC code is
-boilerplate, but that the ``src/Makefile`` is generated according to the
-support modules this Generic IOC contains. You can go and take a look at
-the Makefile and see that it contains ``dbd`` and ``lib`` references for each
-of the support modules in the container.
-See ``/epics/ioc-adsimdetector/ioc/iocApp/src/Makefile``
-
-You will note that the ``ioc`` folder is greyed out in the VSCode explorer. This
-is because it is in ``.gitignore`` and it is purely generated code. If you
-particularly needed to customize the contents of the IOC source tree then
-you can remove it from ``.gitignore`` and commit your changes to the repo. These
-changes would then always get loaded for every instance of the Generic IOC.
+  cd /epics/ioc
+  make
+
+It is useful to understand that /epics/ioc is a soft link to the IOC source
+that came with your generic IOC source code. Therefore if you edit this
+code and recompile it, the changes will be visible inside the container and
+outside the container. Meaning that the repository ``ioc-adsimdetector`` is
+now showing your changes and you could push them up to GitHub if you wanted.
+
+The above is true because your project folder ioc-adsimdetector is mounted into
+the container's filesystem with a bind mount at the same place that the
+ioc files were originally placed by the container build.
+
+epics-containers devcontainers have carefully curated host filesystem mounts
+that allow them to look as similar as possible to the runtime container but
+at the same time preserve any changes that you make in the host file system.
+This is important because the container filesystem is temporary and will be
+destroyed when the container is rebuilt or deleted.
+
+The IOC code is entirely boilerplate boilerplate, the ``src/Makefile``
+determines which dbd and lib files to link by including two files that
+ibek generated during the container build namely
+``/epics/support/configure/lib_list`` and ``/epics/support/configure/dbd_list``.
+
+You can go and take a look at the Makefile in
+``/epics/ioc/iocApp/src/Makefile`` to see how this is done.
 
 The Generic IOC should now be ready to run inside of the container. To do this: