Skip to content

Commit 7f28d30

Browse files
committed
update create beamline repo
1 parent e68c45d commit 7f28d30

File tree

4 files changed

+67
-70
lines changed

4 files changed

+67
-70
lines changed

docs/explanations/repositories.md

Lines changed: 14 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -3,25 +3,24 @@
33
:::{note}
44
**DLS Users** DLS is currently using these locations for assets:
55

6-
- Generic IOC Source: `https://github.com/epics-containers<Generic ioc>`
7-
- Beamline Source repos: `https://gitlab.diamond.ac.uk/controls/containers/beamline/<beamline>`
8-
- Accelerator Source repos: `https://gitlab.diamond.ac.uk/controls/containers/accelerator/<domain>`
9-
- Generic IOC Container Images: `ghcr.io/epics-containers/<Generic ioc>`
10-
- epics-containers Helm Charts: `https://github.com/orgs/epics-containers/packages?repo_name=ec-helm-charts`
6+
- Public Generic IOC Source: <https://github.com/epics-containers/>
7+
- Private Generic IOC Source: <https://gitlab.diamond.ac.uk/controls/containers/iocs>
8+
- Beamline Source repos: <https://gitlab.diamond.ac.uk/controls/containers/beamline/>
9+
- Accelerator Source repos: <https://gitlab.diamond.ac.uk/controls/containers/accelerator/>
10+
- Generic IOC Container Images:<ghcr.io/epics-containers/>
11+
- epics-containers Helm Charts:<https://github.com/orgs/epics-containers/packages?repo_name=ec-helm-charts>
1112
:::
1213

1314
## Where to Keep Source Code
1415

1516
There are two main kinds of source repositories used in epics-containers:
1617

1718
- Generic IOC Source
18-
- Beamline / Accelerator Domain IOC Instance Source
19+
- Beamline / Accelerator Services Repositories for IOC instances and other services.
1920

2021
### Generic IOC Source Repositories
2122

22-
For Generic IOCs it is recommended that these be stored in public repositories
23-
on GitHub. This allows the community to benefit from the work of others and
24-
also contribute to the development of the IOC.
23+
For public Generic IOC container images, the GitHub Container Registry is a good choice. It allows the containers to live at a URL related to the source repository that generated them. The default ioc-template comes with Github Actions that build the container and push it to the GitHub Container Registry.
2524

2625
The intention is that a Generic IOC container image is a reusable component
2726
that can be used by multiple IOC instances in multiple domains. Because
@@ -36,32 +35,24 @@ Integration files for Generic IOCs work with GitHub actions, but also
3635
can work with DLS's internal GitLab instance (this could be adapted for
3736
other facilities' internal GitLab instances or alternative CI system).
3837

39-
### IOC Instance Domain Repositories
38+
### IOC Services Repositories
4039

4140
These repositories are very much specific to a particular domain or beamline
4241
in a particular facility. For this reason there is no strong reason to make
4342
them public, other than to share with others how you are using epics-containers.
4443

45-
At DLS we have a private GitLab instance and we store our domain Repositories
44+
At DLS we have a private GitLab instance and we store our Services Repositories
4645
there.
4746

4847
The CI for domain repos works both with GitHub actions and with DLS's internal
4948
GitLab instance (this could be adapted for
5049
other facilities' internal GitLab instances or alternative CI system).
5150

52-
### BL45P
51+
### p45-services
5352

54-
The test/example beamline at DLS for epics-containers is BL45P.
55-
The domain repository for this
56-
is at <https://github.com/epics-containers/bl45p>. This will always be
57-
kept in a public repository as it is a live example of a domain repo.
53+
The test/example beamline at DLS for epics-containers is p45.
54+
The domain repository for this is at <https://github.com/epics-containers/p45-services>. This will always be kept in a public repository as it is a live example of a domain repo.
5855

59-
## Where to put Registries
56+
This beamline is deployed to Kubernetes at DLS using Argo CD continuous deployment. The repository containing the Argo CD apps that control the deployment is at <https://github.com/epics-containers/p45-deployment>
6057

61-
### Generic IOC Container Images and Source Repos
6258

63-
Usually GHCR but internal supported for license e.g. Nexus Repository Manager
64-
65-
### IOC Instance Domain Repos
66-
67-
Internal git registry or private GitHub registry.

docs/images/copy_gh_repo_addr.png

33.9 KB
Loading

docs/tutorials/create_beamline.md

Lines changed: 42 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -4,88 +4,84 @@
44

55
In this tutorial we will create a new {any}`services-repo`.
66

7-
All IOC Instances that we deploy will be grouped into repositories that define a set of IOC and service instances. Typically each beamline would have its own repository and the accelerator would be split by location or technical area.
7+
All IOC Instances that we deploy will be grouped into repositories that define a set of IOC and service instances. Typically each beamline would have its own repository and the accelerator would be split by technical area.
88

99
In the case of Beamlines, the repo is named after the beamline itself. At DLS
10-
we use the naming convention `blxxc` where `xx` is the beamline number,
11-
and c is the class of beamline.
10+
we use the naming convention `cxx` where `xx` is the beamline number,
11+
and c is the class of beamline. Our naming convention for beamline services repositories is
12+
cxx-services.
1213

1314
:::{note}
1415
You may choose your own naming convention, but lower case letters,
15-
numbers and hyphens only are required for both domain names and
16+
numbers and hyphens are the only characters allowed for both domain names and
1617
IOC names. This is a restriction that helm introduces for package names.
1718
:::
1819

19-
Here we are going to create the test beamline repository `bl01t`. When the project `bl01t` is pushed to GitHub, continuous integration (CI) will verify each of the IOC instances.
20+
Here we are going to create the test beamline repository `t01-services`. When the project is pushed to GitHub, continuous integration (CI) will verify each of the IOC instances.
2021

2122
This beamline repository will be made from a template that comes with a single example IOC and further steps in the following tutorials will teach you how to add your own.
2223

2324
:::{note}
24-
If you are going to work in a shared environment then it is worth choosing
25-
a different name for your test beamline and its associated PV prefixes. PVs
26-
will be published on the local subnet by default so two people working on the
27-
tutorials without doing this will clash. If you do this just remember to
28-
substitute your beamline name for `bl01t` in the following instructions.
25+
If you are working is a shared environment you need not create a unique beamline name or PV prefix because the example runs with all PVs published on localhost only. Your example beamline will be isolated from other users on the same network.
2926
:::
3027

31-
3228
## To Start
3329

34-
For this exercise you will require a github user account or organization in
35-
which to place the new repo. If you do not have one then follow GitHub's
36-
[instructions].
30+
For this exercise you will require a github user account or organization in which to place the new repo. If you do not have one then follow GitHub's [instructions].
3731

3832
Log in to your account by going here <https://github.com/login>.
3933

40-
You will also need to set up ssh keys to authenticate to Github from git. See
41-
[about ssh].
34+
You will also need to set up ssh keys to authenticate to Github from git. See [about ssh].
4235

4336
(create-new-beamline-local)=
4437
## Create a New Beamline Repo for local deployments
4538

46-
Here we will use the ec services template repository to make a new beamline.
39+
Here we will use a services template repository to make a new beamline.
4740

48-
NOTE: for these tutorials we will use your personal GitHub Account to
49-
store everything we do, all source repositories and container images. For
50-
production, each facility will need its own policy for where to store these
51-
assets. See `../explanations/repositories`.
41+
NOTE: for these tutorials we will use your personal GitHub Account to store everything we do, all source repositories and container images. For production, each facility will need its own policy for where to store these assets. See {any}`../explanations/repositories`.
5242

5343
## Steps
5444

55-
1. Go to your GitHub account home page. Click on 'Repositories' and then 'New', give your new repository the name `bl01t` plus a description, then click 'Create repository'.
45+
1. Make sure you have activated the python virtual environment and that `copier` is installed. See instructions here: {any}`ec`.
5646

57-
1. From a command line with your virtual environment activated. Use copier to start to make a new repository like this:
47+
1. Use copier to copy the services template repository to a new repository named `t01-services`. Note that there are two services templates, one for local deployments (using docker compose) and one for deployments to Kubernetes. We will use the local deployment template here.
5848

59-
```bash
60-
pip install copier
61-
# this will create the folder bl01t in the current directory
62-
copier copy gh:epics-containers/ec-services-template --trust bl01t
63-
```
64-
1. Answer the copier template questions as follows:
49+
```bash
50+
copier copy gh:epics-containers/services-template-compose t01-services
51+
```
6552

53+
1. Answer the copier template questions with their default values as follows:
6654

67-
<pre><font color="#5F87AF">🎤</font><b> Where are you deploying these IOCs and services?</b>
68-
<b> </b><font color="#FFAF00"><b>beamline</b></font>
69-
<font color="#5F87AF">🎤</font><b> Short name for the beamline, e.g. &quot;bl47p&quot;, &quot;bl20j&quot;, &quot;bl21i&quot;</b>
70-
<b> </b><font color="#FFAF00"><b>bl01t</b></font>
55+
<pre><font color="#75507B">hgv27681</font>@<font color="#C4A000">pc0116</font>: <font color="#729FCF"><b>/scratch/hgv27681/work</b></font>
56+
$ copier copy gh:epics-containers/services-template-compose t01-services <font color="#4E9A06">[10:47:49]</font>
57+
This template will create a new repository for deploying IOCs and services to
58+
the local machine using docker compose.
59+
60+
<font color="#5F87AF">🎤</font><b> Short name for the collection of services, e.g. &quot;t01&quot;, &quot;p47&quot;, &quot;i20-1&quot;, &quot;i21&quot;</b>
61+
<b> </b><font color="#FFAF00"><b>t01</b></font>
7162
<font color="#5F87AF">🎤</font><b> A One line description of the module</b>
72-
<b> </b><font color="#FFAF00"><b>beamline bl01t IOC Instances and Services</b></font>
73-
<font color="#5F87AF">🎤</font><b> Cluster namespace for these IOCs and services.</b>
74-
<b> </b><font color="#FFAF00"><b>local</b></font>
75-
<font color="#5F87AF">🎤</font><b> Git platform hosting the repository.</b>
76-
<b> </b><font color="#FFAF00"><b>github.com</b></font>
77-
<font color="#5F87AF">🎤</font><b> The GitHub organisation that will contain this repo.</b>
78-
<b> </b><font color="#FFAF00"><b>YOUR_GITHUB_NAME_GOES_HERE</b></font>
79-
<font color="#5F87AF">🎤</font><b> Remote URI of the repository.</b>
80-
<b> </b><font color="#FFAF00"><b>[email protected]:YOUR_GITHUB_NAME/bl01t.git</b></font>
63+
<b> </b><font color="#FFAF00"><b>t01 IOC Instances and Services</b></font>
64+
65+
Copying from template version 3.5.0
66+
<font color="#06989A"> ... </font>
8167
</pre>
8268

69+
1. Create your new repository on GitHub in your personal space by following this link <https://github.com/new>. Give it the name t01-services and a description of "t01 IOC Instances and Services". Then click "Create repository".
70+
71+
Now copy the ssh address of your new repository from the GitHub page.
72+
73+
:::{figure} ../images/copy_gh_repo_addr.png
74+
copying the repository address from GitHub
75+
:::
76+
8377
1. Make the first commit and push the repository to GitHub.
8478

8579
```bash
86-
cd bl01t
80+
cd t01-services
81+
git init -b main
8782
git add .
8883
git commit -m "initial commit"
84+
git remote add origin >>>>paste your ssh address here<<<<
8985
git push -u origin main
9086
```
9187

@@ -104,8 +100,8 @@ You can now give your repository a version tag like this:
104100

105101
```bash
106102
# open a terminal in vscode: Menu -> Terminal -> New Terminal
107-
git tag 2024.3.1
108-
git push origin 2024.3.1
103+
git tag 2024.9.1
104+
git push origin 2024.9.1
109105
```
110106

111107
We use `CalVer` version numbers for beamline repositories and Generic IOCs.
@@ -116,7 +112,7 @@ CalVer is described here: <https://calver.org/> and is used where semantic
116112
versioning is not appropriate because the repository contains a mix of
117113
dependencies and does not have a clear API.
118114

119-
Note that 2024.3.1 represents the time that this tutorial was last updated.
115+
Note that 2024.9.1 represents the date that this tutorial was last updated.
120116
For completeness you could use the current year and month instead. You
121117
are also free to choose your own versioning scheme as this is not enforced by
122118
any of the epics-containers tools.

docs/tutorials/setup_workstation.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,17 @@ The links below have details of how to install your choice of container platform
8080
- [Install docker]
8181
- [Install podman]
8282

83-
The docker install page encourages you to install Docker Desktop. This is a paid for product and is not required for this tutorial. You can install the free linux CLI tools by clicking on the appropriate linux distribution link.
83+
The docker install page encourages you to install Docker Desktop. This is a paid for product and is not required for this tutorial. You can install the free linux CLI tools by clicking on the appropriate linux distribution link under the "Supported Platforms" heading, for simplicity it is easiest to use the option "Install using the convenience script".
84+
85+
### docker compose for podman users
86+
docker compose allows you to define and run multi-container Docker applications. epics-containers uses it for describing a set of IOCs and other serial services that are deployed together.
87+
88+
If you installed docker using the above instructions then docker compose is already installed. If you installed podman then you will need to install docker compose separately. We prefer to use docker-compose instead of podman-compose because it is more widely and avoids behaviour differences between the two tools. If you are at DLS you just need to run 'module load docker-compose' to get access to docker compose with podman as the back end.
89+
90+
Other users of podman please see these instructions [rootless podman with docker-compose](https://connect.redhat.com/hydra/prm/v1/business/companies/0ed5e6899bce415b89d82cb334da214a/linked-resources/aa9ae6ada5f04000a66472cc0fc18160/content/public/view)
91+
92+
93+
```bash
8494

8595
(python-setup)=
8696

0 commit comments

Comments
 (0)