Err0server

Prerequisites

• git

• NodeJS and NPM: https://nodejs.org/en/download/

• Yarn: npm install -g yarn

• Java JDK 11

• Docker and docker-compose

• PostgreSQL client utilities, version 14

Ultra quick-start Guide

• grab a copy, from develop branch, of the database via gitlab https://gitlab.bluetrail.software/err0-io/err0server/-/pipelines?page=1&scope=all&ref=develop you are looking for junit-tests-with-uat-data:archive and you want to put the err0db-localhost.pg_dump in the root directory of this project.

• in a first terminal window, start the database with docker

err0server$ ./docker-start.sh

• in a second terminal window, restore this database, build all webs, build the jar, start the server:

err0server$ ./reset-db-build-all-web-start-err0server.sh

That’s all folks!

Quick-start Guide

This guide has been tested on:

• MacOS (x86 version)

• Ubuntu (ARM version)

• Ubuntu (x86 version)

Warning

It is not yet possible to run the project in Windows. If you have this environment, please use an Ubuntu Virtual Machine or secondary boot OS.

Step by step:

1. Clone this project locally - remember to have the .ssh key properly setup to access GitLab

   user@host$ git clone git@gitlab.bluetrail.software:err0-io/err0server.git err0server

2. Install git submodules

   err0server$ git submodule init; git submodule update
   err0server$ git submodule foreach git checkout -f

3. Create the DB docker instance

   err0server$ ./docker-start.sh    # if it needs sudo, you need to add your user to the group 'docker', logout and login again.

4. Create the config.json file:

   1. Use config.json.example as a template

   2. Replace lo0 and en0 with your local network devices - you can check them with the ifconfig tool

5. Build front-end apps:

We are using yarn as the package manager, perhaps you will need to set the same version as we use on our build environment, 1.22.19, for each website:

err0server/web/accmgt$ yarn set version 1.22.19
err0server/web/login$ yarn set version 1.22.19
err0server/web/customer$ yarn set version 1.22.19

Before building the webs we need to get the version number from git and gradle:

err0server$ ./gradlew TsBuildConfig # build a 'build config' file in ts with version information

After this, you can build all the web apps with one script: ./build-web.sh…​ or do these one by one!

err0server$ cd web/customer
customer$ yarn install --frozen-lockfile       # once
customer$ ./build.sh         # or, ./dev-build.sh for easier debugging
customer$ cd ..
web$ cd accmgt
accmgt$ yarn install --frozen-lockfile         # once
accmgt$ ./build.sh           # or, ./dev-build.sh for easier debugging
customer$ cd ..
web$ cd login
login$ yarn install --frozen-lockfile         # once
login$ ./build.sh            # or, ./dev-build.sh for easier debugging

+ . Build the back-end

+

err0server$ ./gradlew build

+ Important! Tests with UAT data might fail but the jar will still be executed . Start the server:

+

err0server$ ./start-err0server.sh

+ . Try to access the applications in your local environmnet .. https://open-source.dev.err0.io:8443/

1. Happy coding!

Note	If you are running within a VM and you want to access the local website within the host, please add the following domains to your hosts file and point them to your VM’s IP address: open-source-dev.err0.io login-dev.err0.io customer-service-dev.err0.io

Troubleshoting

• When running docker within an ARM ubunto on a virtualized environment over an ARM Mac, docker instances might not have internet connection, plase refer to https://superuser.com/questions/1130898/no-internet-connection-inside-docker-containers

Git Usage

git : reminder about the submodules, there are many:

user@host % git pull

The first time:

user@host % git submodule init; git submodule update

Reset submodules to last commit:

user@host % git submodule foreach git checkout -f

Submodules are for:

corefabric light weight SaaS pattern, embeds https://vertx.io

web/accmgt/src/lib, web/customer/src/lib, web/login/src/lib shared react service and controls library.

Also, we are using the git flow branch model on this project.

Recommended git utility: https://www.syntevo.com/smartgit/

Build

To build the app, as an executable jar, first build the angular apps using yarn package manager (npm install -g yarn):

err0server$ cd web/customer
customer$ yarn install       # once
customer$ yarn update        # after you see changes to package.json
customer$ ./build.sh         # or, ./dev-build.sh for easier debugging

(NB. web/accmgt and web/login will also need to be built.)

Development will be better with dev-build.sh as this leaves the javascript and css source code maps in the built product.

err0server$ ./gradlew build

To run the app, after building:

err0server$ ./start-err0server.sh

Or, in an IDE, after building once; you can run the class io.err0.server.Err0 this is the entrypoint for the executable jar.

You will need the database running:

err0server$ ./docker-start.sh

To rebuild dockerland:

err0server$ rm .docker-build
err0server$ ./docker-start.sh

To nuke all containers on your system to free up disk space:

err0server$ ./docker-clean.sh

To reset the database, requires recent postgres client psql:

err0server$ ./restore-initial-localhost-database.sh

Backup your database:

err0server$ ./backup-localhost.sh

Restore this backup:

err0server$ ./restore-localhost.sh

NOTE THAT the web projects are loaded into memory at server startup, so if you change them you will need to rebuild them and also restart the server.

Example web endpoints (on localhost)

1. login endpoint: https://login.dev.err0.io:8443/

2. account management: https://customer-service.dev.err0.io:8443/

3. example client: https://open-source.dev.err0.io:8443/

4. example client, XSLT "noscript" public content, http: http://open-source.dev.err0.io:8080/

5. system dashboard: https://dashboard.dev.err0.io:8443/

Example web endpoints (on UAT server)

1. login endpoint: https://login.uat.err0.io:8443/

2. account management: https://customer-service.uat.err0.io:8443/

3. example client: https://open-source.uat.err0.io:8443/

4. example client, XSLT "noscript" public content, http: http://open-source.uat.err0.io:8080/

5. system dashboard: https://dashboard.uat.err0.io:8443/

Example web endpoints (on STAGING server)

1. login endpoint: https://login.staging.err0.io:8443/

2. account management: https://customer-service.staging.err0.io:8443/

3. example client: https://open-source.staging.err0.io:8443/

4. example client, XSLT "noscript" public content, http: http://open-source.staging.err0.io:8080/

5. system dashboard: https://dashboard.staging.err0.io:8443/

UAT Image

It is possible to build an image for UAT, first ensure you have built via gradle:

$ ./gradlew build

Then build the test docker container for the app:

$ ./docker-build-app.sh

Finally, test this on localhost:

$ docker run --publish 8443:8443 --publish 8080:8080 err0_io:err0_app

This is a self-contained starter UAT image, with starter database embedded.

Run the 'with UAT data' unit tests successfully

You will need to download a recent backup of the uat data, as we load some error codes from our open-source-bundle of projects into err0 before making unit tests which need error codes to succeed.

From gitlab CI/CD download backup:archive to have these, as they were loaded by our unit tests.

The main difference between backup:archive and junit-tests-with-uat-data:archive is that the second contains some software created by the unit tests.

In fact, the tests should pass with both of these database backups, but the second doesn’t test 'software creation'…​

That was the quick version, there’s also the slow version!

1. start from the initial database

2. you checked out the project open-source-bundle didn’t you? make sure it is fresh with git checkout -f; git submodule foreach git checkout -f

3. run err0server of course

4. in open-source-bundle you can run the script ./soak-test.pl

5. sit back and wait, approx time = 1 coffee.

Insert error codes into this project before making a release

We have a script!

err0server$ ./err0-insert.sh

This will add error codes that are missing from the err0 instance 'self-service.err0.io' where we add codes to err0.

Use the published client utility:

First checkout your project, create a token, then…​

yourproject$ docker run --network="host" \
    --mount type=bind,source=`pwd`,destination=/mnt \
    err0io/agent:latest \
    /usr/local/bin/err0.sh \
    --token /mnt/<<your-token>>.json \
    --insert /mnt

to run the tool on your local working directory, using the appropriate token, to insert codes.

Afterwards, as a build step you would probably want:

yourproject$ docker run --network="host" \
    --mount type=bind,source=`pwd`,destination=/mnt \
    err0io/agent:latest \
    /usr/local/bin/err0.sh \
    --token /mnt/<<your-token>>.json \
    --analyse --check /mnt

to analyse the codes in the build product, with --check in addition this will fail the build if the error codes are not canonical already, that’s to say there’s a duplicate or a missing error code.

or, in gitlab yaml, using docker:

err0-report:
    image: err0io/agent:latest
    before_script:
        - git fetch --all --tags --force
    script: err0.sh --token <<your-token>>.json --analyse --branch $CI_COMMIT_BRANCH --check .

Let’s encrypt manual SSL certificate renewal.

We have two let’s encrypt environments, one for our production certificate, one for our development certificate.

I have dockerised certbot, and it needs a gandi.ini file with credentials [cf philippe] as it uses the gandi plugin to be able to update the DNS to show ownership to Let’s Encrypt.

The let’s encrypt files are in certbot-prod or certbot (for dev/uat/staging). So, for prod, you would use a "hotfix" branch via git-flow and run these commands:

% ./certbot-prod.sh # to make the certbot image and log in
# cd /certbot-prod
# ./renew.sh # if asked for a password it is password
# ^C
$ sudo chown -R user:user certbot-prod # use your username
$ cd certbot-prod
$ ./makejks.sh
$ cd ..
$ cp certbot-prod/keystore.jks ssl/prod/keystore.jks
$ git add .
$ git commit

Dev/UAT/staging is the same except use certbot as the directory and ssl/keystore.jks as the destination to copy the keystore out of the certbot directory.

REACT dev server

You will need to edit your config.json if you don’t have one copy config.json.example and edit the copy.

The react dev server can be used with one of the projects in the web/ folder at once, just choose a folder name, it goes in the site variable here, the example is for the customer microsite - tenant space. Edit the section for err0-web-server to match this:

    "err0-web-server":[
      {
        "proxy": {"site": "customer", "origin": "localhost", "port": 3000}
      }
    ]

After you have made this change (note you must also have successfully built all the webs), you can:

err0server$ cd web/customer
customer$ npm start # launches a server on localhost 3000

Once this is running you can browse the regular URL e.g. https://open-source.dev.err0.io:8443/ BUT FIRST you must go into your browser settings and delete all site data for all time. You can choose to keep your cookies.

This is because our regular environment deploys Progressive Web App versions of our microsites to devices and they have a strong cache policy :) Check the Dev Tools > Application tab for more information. You can also manually remove the PWA, for example if you are using the React dev server and you see this, you must remove it.

Once removed / Site data cleared, just the F5 reload is needed to get your changes after each save.

Sorry, it is not yet automatic, we need to proxy websockets @ /ws on the server for this to work.

REACT dev server ports map

customer port 3000

knowledgebase port 3001

accmgt port 3002

login port 3003

dashboard port 3004