Skip to content

Commit c309645

Browse files
author
Dennis Labordus
committed
Updating and adding pages.
Signed-off-by: Dennis Labordus <[email protected]>
1 parent 94276a0 commit c309645

File tree

4 files changed

+364
-1
lines changed

4 files changed

+364
-1
lines changed

‎docs/DEPENDABOT.md‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
<!--
2+
SPDX-FileCopyrightText: 2021 Alliander N.V.
3+
4+
SPDX-License-Identifier: CC-BY-4.0
5+
-->
6+
7+
## Dependabot
8+
9+
### Configure Dependabot Settings
10+
11+
The first step is that we want to enable Alerts from Dependabot. This needs to be done for every repository that
12+
uses Dependabot. Go to the settings of the repository and select "Security & analysis". Next enable "Dependabot alerts"
13+
and "Dependabot security updates".
14+
15+
Also remember to give the repository access to the secret DB_GITHUB_PACKAGES at organization level.
16+
17+
### Configure Dependabot (yaml)
18+
19+
Next step is to add a configuration file to the repository that will enable Dependabot. In the repository create a
20+
file "dependabot.yml" in the directory ".github". The basic content will be something like
21+
22+
```yaml
23+
version: 2
24+
25+
registries: #(1)
26+
maven-github:
27+
type: maven-repository
28+
url: https://maven.pkg.github.com/com-pas/*
29+
username: OWNER
30+
password: ${{ secrets.DB_GITHUB_PACKAGES }}
31+
32+
updates:
33+
# Maintain dependencies for GitHub Actions
34+
- package-ecosystem: "github-actions" #(2)
35+
directory: "/"
36+
schedule:
37+
interval: "daily"
38+
open-pull-requests-limit: 5
39+
40+
# Maintain dependencies for Maven
41+
- package-ecosystem: "maven" #(3)
42+
directory: "/"
43+
registries:
44+
- maven-github
45+
schedule:
46+
interval: "daily"
47+
open-pull-requests-limit: 5
48+
```
49+
50+
- (1): This configures the GitHub Maven Repository that Dependabot can use to search for new versions of CoMPAS Dependencies.
51+
- (2): This will check if the newest versions of GitHub Actions are used. Default it will scan the directory ".githuib/workflows"
52+
under the directory defined here (in this case "/").
53+
- (3): This will scan the Maven Plugins and Dependencies defined in the pom.xml files. The GitHub Maven Repository is also
54+
defined here.
55+
56+
Both scans are executed daily and create a maximum of 5 pull request. After a pull request is handled it will create the
57+
next one if there is one.
58+
59+
The configuration can be fine-tuned further. For instance with some libraries it isn't possible to use the latest version.
60+
Below is an example how to prevent Dependabot from creating pull request for these dependencies. In the example
61+
the JAXB Implementation can't higher.
62+
63+
```yaml
64+
# Maintain dependencies for Maven
65+
- package-ecosystem: "maven"
66+
directory: "/"
67+
registries:
68+
- maven-github
69+
schedule:
70+
interval: "daily"
71+
open-pull-requests-limit: 5
72+
ignore:
73+
# Next two dependencies shouldn't be upgrade, because RestEasy isn't using newer version. (2.3.X)
74+
- dependency-name: jakarta.xml.bind:jakarta.xml.bind-api
75+
versions: [ "[3.0,)" ]
76+
- dependency-name: com.sun.xml.bind:jaxb-impl
77+
versions: [ "[3.0,)" ]
78+
```
79+
80+
Dependabot uses the configuration found in the default branch (often 'develop'), so to make it effective use a
81+
pull request to merge it into the default branch.
82+
83+
### Adding Dependabot Secret DB_GITHUB_PACKAGES
84+
85+
Tot access GitHub Packages a secret DB_GITHUB_PACKAGES needs to be created.
86+
- First create a new personal access token from https://github.com/settings/tokens. Tokens can only be created as personal tokens.
87+
The token also must have the right "read:packages".
88+
- Next create a new organisation secret from https://github.com/organizations/com-pas/settings/secrets/dependabot with the value of
89+
personal access token you created above. Name the secret DB_GITHUB_PACKAGES. Make it available to the repositories that have
90+
Dependabot configured.
91+
92+
Now Dependabot can use this secret to access GitHub Packages.
93+
94+
### Handling the pull request
95+
96+
Pull request created by Dependabot can be handled just like other pull request, but there is 1 issue to know.
97+
Some GitHub Actions, like SonarCloud and AutomateProjects, will fail if they are started by the pull request from
98+
Dependabot. This is caused by a security issues that was fixed. These actions can't access Secrets when started by a Bot.
99+
For some of these actions it maybe solved in some way, but if that is not possible just manually re-run the action.
100+
The action will then succeed.

‎docs/DEPLOYMENT.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
<!--
2+
SPDX-FileCopyrightText: 2021 Alliander N.V.
3+
4+
SPDX-License-Identifier: CC-BY-4.0
5+
-->
6+
7+
## Deployment of CoMPAS
8+
We deploy the (native) Docker image of all CoMPAS services to Docker Hub. This way, it can be pulled and deployed into environments of your choice (OpenShift for example).
9+
10+
### Quick Deployment instructions (under construction)
11+
The following instructions are terminal instructions for publishing a Quarkus docker image to Docker Hub. This should be done by a Github Action in the future.
12+
13+
```
14+
docker login # Creating a Docker session
15+
```
16+
In order to publish it to Docker Hub, you have to login first and create a session.
17+
18+
```
19+
./gradlew clean build -Pnative -Dquarkus.native.container-build=true -Dquarkus.container-image.build=true -Dquarkus.container-image.group=group -Dquarkus.container-image.name=name -Dquarkus.container-image.push=true
20+
```
21+
Few points in this single command:
22+
- `-Pnative` creates a native executable (this needs some prework, see the Building Native Image [documentation](https://quarkus.io/guides/building-native-image)).
23+
- `-Dquarkus.native.container-build=true` creates a Linux executable without GraalVM being installed (only necessary if GraalVM is not installed).
24+
- `-Dquarkus.container-image.build=true` creates a container image with the native executable in it.
25+
- `-Dquarkus.container-image.group=group` defining the group name. Standard it's the system user name.
26+
- `-Dquarkus.container-image.name=name` defining the name of the image. Standard it's 'app'.
27+
- `-Dquarkus.container-image.push=true` deploys to docker image to Docker Hub.
28+
29+
## Sources
30+
#### Full documentation about deploying Quarkus application to Docker Hub
31+
https://dev.to/marcuspaulo/tutorial-publish-a-quarkus-application-in-kubernetes-minikube-and-dockerhub-36nd

‎docs/GITHUB_ACTIONS.md‎

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
<!--
2+
SPDX-FileCopyrightText: 2021 Alliander N.V.
3+
4+
SPDX-License-Identifier: CC-BY-4.0
5+
-->
6+
7+
## Github Actions
8+
Every repository within the CoMPAS Github organization need a default set of Github Actions.
9+
[Github Actions](https://github.com/features/actions) are CI/CD steps within a Github Repository that you can configure. This way, you can ensure that certain steps (like building) are always triggered on for example a commit push.
10+
11+
Within CoMPAS, we define the following 'must have' Github Actions:
12+
- [Building](#building)
13+
- [REUSE check](#reuse-check)
14+
- [SonarCloud](#sonarcloud)
15+
- [Docker Hub Deployment](#docker-hub-deployment)
16+
17+
More to follow.
18+
19+
Github Actions are configured using YAML files. These files are stored in the `.github/workflows` directory of a specific repository.
20+
21+
### Building
22+
All source code repositories need some kind of building step.
23+
By default, all source code repositories use Maven as the build tool.
24+
25+
This building step is pretty easy to configure. Just create a `maven_build.yml` file in the `.github/workflows` directory containing the following source code:
26+
27+
```yaml
28+
name: Maven Build
29+
30+
on: push #(1)
31+
32+
jobs:
33+
build:
34+
name: Build
35+
runs-on: ubuntu-latest
36+
timeout-minutes: 15
37+
38+
steps:
39+
- uses: actions/checkout@v2
40+
- name: Set up JDK 1.11
41+
uses: actions/[email protected]
42+
with:
43+
distribution: 'zulu'
44+
java-version: '11'
45+
- name: Create custom Maven Settings.xml #(2)
46+
uses: whelk-io/maven-settings-xml-action@v18
47+
with:
48+
output_file: custom_maven_settings.xml
49+
servers: '[{ "id": "github-packages-compas", "username": "OWNER", "password": "${{ secrets.GITHUB_TOKEN }}" }]'
50+
- name: Build with Maven
51+
run: mvn -s custom_maven_settings.xml -B clean verify #(3)
52+
```
53+
54+
A few points to remember:
55+
- (1): By default, all actions are triggered on a push action.
56+
- (2): This step creates a custom settings.xml for authenticating for Github Packages. For more information,
57+
check the [Contributing file](https://github.com/com-pas/contributing/blob/master/CONTRIBUTING.md).
58+
- (3): This is a default for building a Maven project. It may differ in some cases, please feel free to adjust it.
59+
60+
### REUSE check
61+
For keeping our copyright and licensing information up to date and correct, we use [REUSE](https://reuse.software/) to check this. This is also configured for every separate repository in an easy manner: just create a `reuse.yml` file in the `.github/workflows` directory containing the following source code:
62+
63+
```yaml
64+
name: REUSE Compliance Check
65+
66+
on: push #(1)
67+
68+
jobs:
69+
test:
70+
runs-on: ubuntu-latest
71+
steps:
72+
- uses: actions/checkout@v2
73+
- name: REUSE Compliance Check
74+
uses: fsfe/reuse-action@v1
75+
```
76+
77+
A few points to remember:
78+
- (1): By default, all actions are triggered on a push action.
79+
80+
This is the only thing that has to be done. After this, it will be checked on every push.
81+
82+
#### REUSE badge
83+
For transparancy, CoMPAS repositories also include a REUSE badge in their README for fast checking the REUSE compliance.
84+
85+
Two steps are needed to get a REUSE badge to work:
86+
87+
1. Register the Repository at the [REUSE website](https://api.reuse.software/register). For name and email, check the [Slack channel](https://app.slack.com/client/TLU68MTML).
88+
2. Add the following code to the README:
89+
90+
```md
91+
[![REUSE status](https://api.reuse.software/badge/github.com/com-pas/repoName)](https://api.reuse.software/info/github.com/com-pas/repoName)
92+
```
93+
94+
There is one steps left: Replace `repoName` with the name of the specific repository (as stated in the URL).
95+
96+
After doing all these steps, everything is set up for the REUSE check.
97+
98+
### SonarCloud
99+
For static code analysis, CoMPAS is using [SonarCloud](https://sonarcloud.io/). To configure this, there are several steps that needs to be done.
100+
101+
1. Go to the [CoMPAS Github organization settings](https://github.com/organizations/com-pas/settings/profile), and click on "Installed Github Apps". SonarCloud is listed here already (because we are already using it). Click on the 'configure' button next to it.
102+
2. In the "Repository access" section, select the repository you want to add. By default, not every repository is added as an extra check.
103+
3. Create a new project in [SonarCloud](https://sonarcloud.io/projects/create).
104+
4. Select the repository to be analyzed, click Set Up.
105+
5. Choose the Analysis Method "With Github Actions".
106+
6. It first tells you to create a SONAR_TOKEN secret in your repo. Go to your repository -> Settings - Secrets -> New repository secret -> Name: SONAR_TOKEN. Value: Copy the value from the SonarCloud website into here. Then save the secret
107+
7. Select Maven as the option that best describes our build and **remember the projectKey**. and create a `sonarcloud_analysis.yml` file in the `.github/workflows` directory containing the following source code running.
108+
109+
```yaml
110+
name: SonarCloud Analysis
111+
112+
on: push #(1)
113+
114+
jobs:
115+
build:
116+
name: Build
117+
runs-on: ubuntu-latest
118+
timeout-minutes: 15
119+
120+
steps:
121+
- uses: actions/checkout@v2
122+
with:
123+
fetch-depth: 0
124+
- name: Set up JDK 1.11
125+
uses: actions/[email protected]
126+
with:
127+
distribution: 'zulu'
128+
java-version: '11'
129+
- name: Cache SonarCloud packages
130+
uses: actions/[email protected]
131+
with:
132+
path: ~/.sonar/cache
133+
key: ${{ runner.os }}-sonar
134+
restore-keys: ${{ runner.os }}-sonar
135+
- name: Cache Maven packages
136+
uses: actions/[email protected]
137+
with:
138+
path: ~/.m2
139+
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
140+
restore-keys: ${{ runner.os }}-m2
141+
- name: Create custom Maven Settings.xml #(2)
142+
uses: whelk-io/maven-settings-xml-action@v18
143+
with:
144+
output_file: custom_maven_settings.xml
145+
servers: '[{ "id": "github-packages-compas", "username": "OWNER", "password": "${{ secrets.GITHUB_TOKEN }}" }]'
146+
- name: Build and analyze
147+
env:
148+
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
149+
run: | #(3)
150+
mvn -s custom_maven_settings.xml -B -Psonar \
151+
-Dsonar.projectKey=<insert project key> \
152+
-Dsonar.organization=com-pas \
153+
-Dsonar.host.url=https://sonarcloud.io \
154+
verify org.sonarsource.scanner.maven:sonar-maven-plugin:sonar
155+
```
156+
157+
A few points to remember:
158+
- (1): By default, all actions are triggered on a push action.
159+
- (2): Creates a custom `settings.xml` having the credentials for the Github Packages.
160+
For more information, check our [Contributing](https://github.com/com-pas/contributing/blob/master/CONTRIBUTING.md).
161+
- (3): Replace the `<insert project key>` with the project key you copied.
162+
163+
Once this is set, it's all done!
164+
165+
### Docker Hub Deployment
166+
For automatic deployment of our microservices, CoMPAS uses Docker Hub as the central docker image repository. This way, all Docker images can be pulled from a central image repository.
167+
168+
This step is easy to configure. Just create a `dockerhub_deployment.yml` file in the `.github/workflows` directory containing the following source code:
169+
170+
```yaml
171+
name: Docker Hub Deployment
172+
173+
on:
174+
release:
175+
types: [released] #(1)
176+
177+
jobs:
178+
push_to_registry:
179+
name: Build and publish
180+
runs-on: ubuntu-latest
181+
steps:
182+
- uses: actions/checkout@v2
183+
- name: Login to Docker Hub #(2)
184+
uses: docker/login-action@v1
185+
with:
186+
username: ${{ secrets.DOCKER_HUB_USERNAME }}
187+
password: ${{ secrets.DOCKER_HUB_PASSWORD }}
188+
- name: Extract tag name #(3)
189+
id: extract_tagname
190+
shell: bash
191+
# Extra the tagname form the git reference, value of GITHUB_REF will be something like refs/tags/<tag_name>.
192+
run: echo "##[set-output name=tagname;]$(echo ${GITHUB_REF##*/})"
193+
- name: Set up JDK 11
194+
uses: actions/[email protected]
195+
with:
196+
distribution: 'zulu'
197+
java-version: '11'
198+
- name: Create custom Maven Settings.xml
199+
uses: whelk-io/maven-settings-xml-action@v18
200+
with:
201+
output_file: custom_maven_settings.xml #(4)
202+
servers: '[{ "id": "github-packages-compas", "username": "OWNER", "password": "${{ secrets.GITHUB_TOKEN }}" }]'
203+
- name: Set version with Maven
204+
run: mvn -B versions:set -DprocessAllModules=true -DnewVersion=${{ steps.extract_tagname.outputs.tagname }}
205+
- name: Deploy with Maven to GitHub Packages and Docker Hub #(5)
206+
run: ./mvnw -B -s custom_maven_settings.xml -Prelease,native clean deploy
207+
```
208+
209+
A few points to remember:
210+
- (1): By default, the docker image is only deployed on release. This way we have a strict deployment flow, and versions always have the same content. For more information about types of releases, check the [Github Actions documentation](https://docs.github.com/en/actions/reference/events-that-trigger-workflows#release).
211+
- (2): Before deploying to Docker Hub, we need to login first. This can be done by executing a GitHub Action. In this example, `DOCKER_HUB_USERNAME` and `DOCKER_HUB_PASSWORD` are used, which are secrets stored at [CoMPAS organization](https://github.com/organizations/com-pas/settings/secrets/actions). For more information about the username and password, ask in the the [Slack channel](https://app.slack.com/client/TLU68MTML).
212+
- (3): Extract the tag name from Git and use that as version to be set with Maven (${{ steps.extract_tagname.outputs.tagname }}).
213+
- (4): Creates a custom `settings.xml` having the credentials for the Github Packages.
214+
For more information, check our [Contributing](https://github.com/com-pas/contributing/blob/master/CONTRIBUTING.md).
215+
- (5): Building and publishing the docker image is build tool / framework specific. By default, CoMPAS services are using Quarkus and Maven. Deploying to Docker Hub is quite easy using Quarkus and maven, it's just a matter of building in combination with setting some properties. In this example, we use the `quarkus-profile` parameter instead of including all the parameters. This way, we can define profile specific properties in our `application.properties` file (For more information about this, check our [Docker Hub Deployment page](./DEPLOYMENT.md)):
216+
217+
```ini
218+
%publishNativeImage.quarkus.native.container-build=true
219+
%publishNativeImage.quarkus.container-image.build=true
220+
%publishNativeImage.quarkus.container-image.group=lfenergycompas
221+
%publishNativeImage.quarkus.container-image.name=compas-scl-data-service
222+
%publishNativeImage.quarkus.container-image.push=true
223+
```

‎docs/_includes/sidebar.html‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,18 @@ <h2>
2020
<a class="sidebar-nav-item{% if page.url == 'GOVERNANCE.html' %} active{% endif %}" href="{{ site.baseurl }}/GOVERNANCE.html">Governance</a>
2121
<a class="sidebar-nav-item{% if page.url == 'DEVELOPING.html' %} active{% endif %}" href="{{ site.baseurl }}/DEVELOPING.html">Developing</a>
2222
<ul class="nav-list ">
23-
<li class="nav-list-item ">
23+
<li class="nav-list-item">
2424
<a class="sidebar-nav-item{% if page.url == 'STYLEGUIDE.html' %} active{% endif %}" href="{{ site.baseurl }}/STYLEGUIDE.html">Styleguide</a>
2525
</li>
26+
<li class="nav-list-item">
27+
<a class="sidebar-nav-item{% if page.url == 'GITHUB_ACTIONS.html' %} active{% endif %}" href="{{ site.baseurl }}/GITHUB_ACTIONS.html">Github Actions</a>
28+
</li>
29+
<li class="nav-list-item">
30+
<a class="sidebar-nav-item{% if page.url == 'DEPENDABOT.html' %} active{% endif %}" href="{{ site.baseurl }}/DEPENDABOT.html">Dependabot</a>
31+
</li>
32+
<li class="nav-list-item">
33+
<a class="sidebar-nav-item{% if page.url == 'DEPLOYMENT.html' %} active{% endif %}" href="{{ site.baseurl }}/DEPLOYMENT.html">Deployment</a>
34+
</li>
2635
</ul>
2736

2837
</nav>

0 commit comments

Comments
 (0)