|
| 1 | +# Documentation |
| 2 | + |
| 3 | +This piece of documentation aims to help DevOps to make changes and improvements for the Possible-X project. |
| 4 | + |
| 5 | +It has three major sections. The first section will be about the used technologies and approaches. |
| 6 | +The second part focuses on the details of the deployment in the Development environment and the Integration Environment. |
| 7 | +The final part is a list of suggested improvements and things to consider for future development. |
| 8 | + |
| 9 | +## Basic Building Blocks |
| 10 | + |
| 11 | +Currently the setup for Possible-X consists of the following parts: |
| 12 | + |
| 13 | +- EDC Extension |
| 14 | +- EDC Participant Portal |
| 15 | +- Main Portal |
| 16 | +- Creation Wizard API |
| 17 | +- DAPS Service |
| 18 | +- DID Service |
| 19 | +- Fraunhofer Catalog, which again consists of |
| 20 | + - Piveau Hub Repo |
| 21 | + - Piveau Hub Search |
| 22 | + - Nomralization Service |
| 23 | + - Piveau UI Service |
| 24 | + - Elastic Search |
| 25 | + - Virtuoso |
| 26 | + |
| 27 | +For details about the individual projects, please consult their git repos. |
| 28 | + |
| 29 | +Beyond these basic blocks we introduced some tools for monitoring, convenience and to improve the development experience: |
| 30 | + |
| 31 | +- Grafana |
| 32 | +- Prometheus |
| 33 | +- Loki |
| 34 | +- PGAdmin |
| 35 | +- Flux |
| 36 | + |
| 37 | + |
| 38 | +### A bit of history |
| 39 | + |
| 40 | +From the start we planned to utilize Flux as GitOps tool. What was not entirely hammered out yet was the way we deploy the individual services. |
| 41 | +So we started with Helm Charts, as the edc deployment already was deployed that way. |
| 42 | + |
| 43 | +Later we decided to use kustomize for the integratin environment. This difference was never synced back completly to the dev environment. Although, later developments like some common resources (did, catalog, etc.). |
| 44 | + |
| 45 | +## Environments |
| 46 | + |
| 47 | +We have essentially two environments. Once we had a similar environment for presentation purposes, that was called poc/mvd/demo environment. THis env was discontinued after the Integration environment was setup. |
| 48 | + |
| 49 | +### Development Environment |
| 50 | + |
| 51 | +#### Deployment |
| 52 | + |
| 53 | +The catalog and the did service are deployed via Flux. All other applications are deployed in a similar way using helm charts. |
| 54 | + |
| 55 | +The basic setup is similar for all of these installations: |
| 56 | + |
| 57 | +#### Github Action |
| 58 | + |
| 59 | +You will find a ci.yaml file in the .github/workflows directory. |
| 60 | +Each workflow will setup a docker image, in which the code the build and tests are run. |
| 61 | +At the end of those steps the image is pushed towards the github container registry and can be used. |
| 62 | +The final step of the pipeline (at least for develop) is the push of the new image towards the environment. |
| 63 | +This is done within the final action of the pipeline. |
| 64 | + |
| 65 | +As mentioned the setup uses a dockerimage, which can usually be found within the backend directory. In any case the exact file path is referenced within the github actions. |
| 66 | + |
| 67 | +The Dockerfile usually has two images - one for building and testing of the code and one for the actual deployment. The later will extract the relevant files from the former. |
| 68 | +One special case is the participant portal - which has a small script within the image generation to install a headless chrome. |
| 69 | +Why do we need this? The Participant Portal utilizes Browser Tests using the karma tooling. A special caviat is that this does not work out of the box, by simply installing a chrome. If one does that, the chrome will crash, as it is run as root and that would require a special flag called `no-sandbox`. Luckily this can be configured within the karma.config.js: |
| 70 | + |
| 71 | +```js |
| 72 | + browsers: ['ChromeHeadlessNoSandbox'], |
| 73 | + customLaunchers: { |
| 74 | + ChromeHeadlessNoSandbox: { |
| 75 | + base: 'ChromeHeadless', |
| 76 | + flags: ['--no-sandbox'] |
| 77 | + } |
| 78 | + } |
| 79 | +``` |
| 80 | + |
| 81 | +#### Helm Charts |
| 82 | + |
| 83 | +The helm charts can be found in the deployment/helm directory. The charts are homegrown and should be rather straigth forward (or as straight forward as go templates go). |
| 84 | +The values are usually located directly next to helm directory, e.g. in a dev directory. Also additional sealed secrets are stored there. |
| 85 | +An exception to rule is the configuration of the consumer and provider EDCs. It includes some secret data, that should not be comited to a public repository. Hence, to this date the required values are stored in an offline repository, that is available amon g the developers. |
| 86 | + |
| 87 | +### Integration Environment |
| 88 | + |
| 89 | + |
| 90 | + |
| 91 | +### Additional Tools |
| 92 | + |
| 93 | +#### Grafana |
| 94 | + |
| 95 | + |
| 96 | + |
| 97 | +#### PG Admin |
| 98 | + |
| 99 | + |
| 100 | +## Future Improvements |
| 101 | + |
| 102 | +Here is a list of things that may improve the overall |
| 103 | + |
| 104 | +- Remove the Helm Charts entirely - it would be good to migrate the Dev Deployment Process to |
| 105 | + |
| 106 | + |
0 commit comments