Skip to content

Commit 95ac2d1

Browse files
author
Dennis Labordus
committed
Added PostgreSQl version of Docker Compose and Testing Github Action.
Signed-off-by: Dennis Labordus <[email protected]>
1 parent 5f2cb87 commit 95ac2d1

File tree

5 files changed

+247
-13
lines changed

5 files changed

+247
-13
lines changed

‎.github/workflows/run-integration-tests.yml‎ renamed to ‎.github/workflows/run-integration-tests-basex.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
#
33
# SPDX-License-Identifier: Apache-2.0
44

5-
name: Run integration tests
5+
name: Run integration tests (BaseX)
66

77
on:
88
push:
@@ -31,7 +31,7 @@ jobs:
3131
key: ${{ runner.os }}-buildx-${{ hashFiles('**/Dockerfile') }}
3232

3333
- name: Start containers
34-
run: docker-compose -f "compas/docker-compose.yml" up -d --build
34+
run: docker-compose -f "compas/docker-compose-basex.yml" up -d --build
3535
env:
3636
COMPAS_HOSTNAME: compas_reverse-proxy_1
3737
- name: Wait until containers started
@@ -79,4 +79,4 @@ jobs:
7979
8080
- name: Stop containers
8181
if: always()
82-
run: docker-compose -f "compas/docker-compose.yml" down -v
82+
run: docker-compose -f "compas/docker-compose-basex.yml" down -v

‎.github/workflows/run-integration-tests-postgresql.yml‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# SPDX-FileCopyrightText: 2021 Alliander N.V.
2+
#
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
name: Run integration tests (PostgreSQL)
6+
7+
on:
8+
push:
9+
branches:
10+
- '**'
11+
- '!main'
12+
pull_request:
13+
branches:
14+
- 'main'
15+
16+
jobs:
17+
integration-tests:
18+
runs-on: ubuntu-latest
19+
timeout-minutes: 30
20+
21+
steps:
22+
- name: Checkout
23+
uses: actions/checkout@v2
24+
- name: Set up Docker Buildx
25+
id: buildx
26+
uses: docker/setup-buildx-action@v1
27+
- name: Cache Docker Register
28+
uses: actions/cache@v2
29+
with:
30+
path: /tmp/.buildx-cache
31+
key: ${{ runner.os }}-buildx-${{ hashFiles('**/Dockerfile') }}
32+
33+
- name: Start containers
34+
run: docker-compose -f "compas/docker-compose-postgresql.yml" up -d --build
35+
env:
36+
COMPAS_HOSTNAME: compas_reverse-proxy_1
37+
- name: Wait until containers started
38+
run: sh ./bin/docker-wait-on-containers.sh
39+
40+
- name: Created output directory (firefox)
41+
run: mkdir -p target/firefox
42+
- name: Execute integration tests (firefox)
43+
# Using the Docker Image from ppodgorsek (https://github.com/ppodgorsek/docker-robot-framework)
44+
# to run the Robot Framework Test Scripts.
45+
# To connect with the Docker Compose and use http://compas_reverse-proxy_1/ as URL we are
46+
# connecting to the same network "compas_default" that's started bij Docker Compose.
47+
run: |
48+
docker run --rm -e BROWSER=firefox \
49+
-e ROBOT_OPTIONS="-v url:http://compas_reverse-proxy_1/" \
50+
-v $(pwd)/target/firefox:/opt/robotframework/reports:Z \
51+
-v $(pwd)/integration-testing:/opt/robotframework/tests:Z \
52+
--user $(id -u):$(id -g) \
53+
--network=compas_default \
54+
ppodgorsek/robot-framework:latest
55+
56+
# We need to wait for a new version of the Docker Image.
57+
# There is an issue with Chromium (https://github.com/ppodgorsek/docker-robot-framework/issues/372)
58+
# - name: Created output directory (chromium)
59+
# run: mkdir -p target/chromium
60+
# - name: Execute integration tests (chromium)
61+
# run: |
62+
# docker run --rm -e BROWSER=chromium \
63+
# -e ROBOT_OPTIONS="-v url:http://compas_reverse-proxy_1/" \
64+
# -v $(pwd)/target/firefox:/opt/robotframework/reports:Z \
65+
# -v $(pwd)/integration-testing:/opt/robotframework/tests:Z \
66+
# --user $(id -u):$(id -g) \
67+
# --network=compas_default \
68+
# ppodgorsek/robot-framework:latest
69+
70+
- name: Archive Robot Framework Reports
71+
if: failure()
72+
uses: actions/upload-artifact@v2
73+
with:
74+
name: robot-framework-reports
75+
retention-days: 7
76+
path: |
77+
target/firefox/*.*
78+
target/chromium/*.*
79+
80+
- name: Stop containers
81+
if: always()
82+
run: docker-compose -f "compas/docker-compose-postgresql.yml" down -v

‎README.md‎

Lines changed: 39 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -10,16 +10,19 @@ SPDX-License-Identifier: Apache-2.0
1010
# CoMPAS Deployment Repository
1111
Containing tools / configurations for deploying CoMPAS services.
1212

13-
## Docker Compose
14-
There is a pre configured [Docker Compose](compas/docker-compose.yml) file, which starts all the given CoMPAS services.
15-
To start all configured services, go to the `compas` directory and run the following two commands:
13+
## Docker Compose (BaseX)
14+
There is a pre configured [Docker Compose](compas/docker-compose-basex.yml) file, which starts all the given CoMPAS services.
15+
To start all configured services using BaseX, run the following two commands:
1616

1717
```
1818
# Build (if needed) and start all the containers in the background.
19-
docker-compose up -d --build
19+
docker-compose -f compas/docker-compose-basex.yml up -d --build
2020
```
2121

22-
This first command builds 2 containers (keycloak and reverse proxy) and next command starts all CoMPAS services at the same time.
22+
This command will first build the custom images for Keycloak and the Reverse Proxy and then start all containers.
23+
To know if all containers are running execute the shell script "bin/docker-wait-on-containers.sh".
24+
This script will wait until all containers are running.
25+
2326
Now, the following services are available:
2427

2528
- open-scd, available at [http://localhost/](http://localhost/).
@@ -36,16 +39,42 @@ To stop and remove all the containers run the command:
3639

3740
```
3841
# Stop all containers and remove the volumes.
39-
docker-compose down -v
42+
docker-compose -f compas/docker-compose-basex.yml down -v
43+
```
44+
45+
The option '-v' also removes the volumes created, so all data is lost with this option.
46+
47+
## Docker Compose (PostgreSQL)
48+
There is a pre configured [Docker Compose](compas/docker-compose-postgresql.yml) file, which starts all the given CoMPAS services.
49+
To start all configured services using PostgreSQL, run the following two commands:
50+
51+
```
52+
# Build (if needed) and start all the containers in the background.
53+
docker-compose -f compas/docker-compose-postgresql.yml up -d --build
54+
```
55+
56+
This command will first build the custom images for Keycloak and the Reverse Proxy and then start all containers.
57+
To know if all containers are running execute the shell script "bin/docker-wait-on-containers.sh".
58+
This script will wait until all containers are running.
59+
60+
The URLs are the same as for the BaseX version, except of course the BaseX Container. This is now a PostgreSQL
61+
container. The PostgreSQL container is available at port 5432.
62+
63+
To stop and remove all the containers run the command:
64+
65+
```
66+
# Stop all containers and remove the volumes.
67+
docker-compose -f compas/docker-compose-postgresql.yml down -v
4068
```
4169

4270
The option '-v' also removes the volumes created, so all data is lost with this option.
4371

4472
### Known issue with Docker Compose
4573

46-
Using the current configuration with Keycloak and OpenResty (lua-resty-session) sometimes gives an error "**state from argument does not match state restored from session**".
74+
Using the current configuration with Keycloak and OpenResty (lua-resty-session) sometimes gives an error
75+
"**state from argument does not match state restored from session**".
4776
This mostly happens after using logout or the session is expired. The login page is shown, but after login this error is shown.
48-
Just open the url [http://localhost/](http://localhost/) again and OpenSCD is shown again also being logged in.
77+
Just open the url [http://localhost/](http://localhost/) again and OpenSCD is shown again, also being logged in.
4978

5079
## Keycloak Demo Configuration
5180
For demo purposes, a [demo Keycloak configuration](compas/keycloak/keycloak_compas_realm.json) is created which can be imported when
@@ -56,7 +85,8 @@ The following Keycloak attributes have been added:
5685
- **OpenSCD client**: A client has been made for interacting with OpenSCD.
5786
- **SCL Data Service client**: A client has been made for interacting with the SCL Data Service.
5887
- **CRUD roles for the SCL Data Service**: Create, Read, Update and Delete roles have been added to the SCL Data Service client.
59-
When interacting with the SCL Data Service, a JWT token needs to have certain roles before interaction is possible. These roles are assigned to certain users (see below).
88+
When interacting with the SCL Data Service, a JWT token needs to have certain roles before interaction is possible.
89+
These roles are assigned to certain users (see below).
6090
- **CoMPAS Group**: A CoMPAS demo group has been added.
6191
- **A Demo User**: A Demo user without specific roles.
6292
- Username: 'user'

‎compas/docker-compose.yml‎ renamed to ‎compas/docker-compose-basex.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ services:
3939
scl-data-service:
4040
labels:
4141
compas: true
42-
image: "lfenergy/compas-scl-data-service:0.8.3"
42+
image: "lfenergy/compas-scl-data-service:0.9.0-basex"
4343
environment:
4444
- BASEX_HOST=basex
4545
- BASEX_PORT=1984

‎compas/docker-compose-postgresql.yml‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# SPDX-FileCopyrightText: 2021 Alliander N.V.
2+
#
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
version: "3.9"
6+
7+
services:
8+
postgresql:
9+
labels:
10+
compas: true
11+
image: "postgres:latest"
12+
ports:
13+
- "5432:5432"
14+
environment:
15+
- POSTGRES_PASSWORD=postgres
16+
- POSTGRES_DB=compas
17+
- PGDATA=/var/lib/postgresql/data/compas
18+
healthcheck:
19+
test: /usr/bin/pg_isready
20+
interval: 5s
21+
timeout: 10s
22+
retries: 120
23+
volumes:
24+
- postgresql-data-volume:/var/lib/postgresql/data
25+
26+
keycloak:
27+
build:
28+
context: ./keycloak
29+
args:
30+
COMPAS_HOSTNAME: ${COMPAS_HOSTNAME}
31+
labels:
32+
compas: true
33+
ports:
34+
- "8089:8080"
35+
environment:
36+
- KEYCLOAK_FRONTEND_URL=http://${COMPAS_HOSTNAME}/auth/
37+
- KEYCLOAK_HOSTNAME=${COMPAS_HOSTNAME}
38+
- KEYCLOAK_HTTP_PORT=80
39+
- PROXY_ADDRESS_FORWARDING=true
40+
healthcheck:
41+
test: ["CMD", "curl", "-f", "http://localhost:8080/auth/"]
42+
interval: 30s
43+
timeout: 10s
44+
retries: 5
45+
46+
scl-data-service:
47+
labels:
48+
compas: true
49+
image: "lfenergy/compas-scl-data-service:0.9.0-postgresql"
50+
environment:
51+
- POSTGRESQL_HOST=postgresql
52+
- POSTGRESQL_DB=compas
53+
- JWT_VERIFY_KEY=http://keycloak:8080/auth/realms/compas/protocol/openid-connect/certs
54+
- JWT_VERIFY_ISSUER=http://${COMPAS_HOSTNAME}/auth/realms/compas
55+
- JWT_VERIFY_CLIENT_ID=scl-data-service
56+
- JWT_GROUPS_PATH=resource_access/scl-data-service/roles
57+
- USERINFO_NAME_CLAIMNAME=name
58+
- USERINFO_WHO_CLAIMNAME=name
59+
- USERINFO_SESSION_WARNING=20
60+
- USERINFO_SESSION_EXPIRES=30
61+
healthcheck:
62+
test: ["CMD", "curl", "-f", "http://localhost:8080/compas-scl-data-service/q/health/ready"]
63+
interval: 15s
64+
timeout: 10s
65+
retries: 5
66+
depends_on:
67+
- postgresql
68+
- keycloak
69+
70+
cim-mapping:
71+
labels:
72+
compas: true
73+
image: "lfenergy/compas-cim-mapping:0.7.6"
74+
environment:
75+
- JWT_VERIFY_KEY=http://keycloak:8080/auth/realms/compas/protocol/openid-connect/certs
76+
- JWT_VERIFY_ISSUER=http://${COMPAS_HOSTNAME}/auth/realms/compas
77+
- JWT_VERIFY_CLIENT_ID=cim-mapping
78+
- JWT_GROUPS_PATH=resource_access/cim-mapping/roles
79+
- USERINFO_WHO_CLAIMNAME=name
80+
healthcheck:
81+
test: ["CMD", "curl", "-f", "http://localhost:8080/compas-cim-mapping/q/health/ready"]
82+
interval: 15s
83+
timeout: 10s
84+
retries: 5
85+
depends_on:
86+
- keycloak
87+
88+
open-scd:
89+
labels:
90+
compas: true
91+
image: "lfenergy/compas-open-scd:v0.7.1-compas-4"
92+
healthcheck:
93+
test: ["CMD", "curl", "-f", "http://localhost/"]
94+
interval: 30s
95+
timeout: 10s
96+
retries: 5
97+
depends_on:
98+
- scl-data-service
99+
- cim-mapping
100+
101+
reverse-proxy:
102+
labels:
103+
compas: true
104+
build:
105+
context: ./reverse-proxy
106+
args:
107+
COMPAS_HOSTNAME: ${COMPAS_HOSTNAME}
108+
ports:
109+
- "80:80"
110+
healthcheck:
111+
test: ["CMD", "curl", "-f", "http://localhost/"]
112+
interval: 30s
113+
timeout: 10s
114+
retries: 5
115+
depends_on:
116+
- keycloak
117+
- scl-data-service
118+
- cim-mapping
119+
- open-scd
120+
121+
volumes:
122+
postgresql-data-volume:

0 commit comments

Comments
 (0)