Update READMEs, env. variables and more

Author: elpiel

Dockerfile

@@ -19,6 +19,9 @@ FROM ubuntu:20.04
 
 RUN apt update && apt-get install -y libssl-dev ca-certificates
 
+# `production` or `development` - default: `development`
+ENV ENV=
+
 # `ethereum` or `dummy`
 ENV ADAPTER=
 

Dockerfile-sentry

@@ -45,6 +45,16 @@ FROM ubuntu:20.04
 
 RUN apt update && apt-get install -y libssl-dev ca-certificates
 
+# `production` or `development` - default: `development`
+ENV ENV=
+
+# Redis URL - default: `redis://127.0.0.1:6379`
+ENV READIS_URL=
+# The IP address on which the sentry server will be listening - default: `0.0.0.0`
+ENV IP_ADDR=
+# The Port on which the sentry server will be listening - default: `8005`
+ENV PORT=
+
 # `ethereum` or `dummy`
 ENV ADAPTER=
 

README.md

@@ -1,27 +1,20 @@
-# AdEx Validator Stack in Rust [![CI](https://github.com/AdExNetwork/adex-validator-stack-rust/workflows/Continuous%20Integration/badge.svg)](https://github.com/AdExNetwork/adex-validator-stack-rust/actions)
-
-The Rust implementation of the Validator Stack
-
-Reference implementation of the [AdEx validator stack](https://github.com/adexnetwork/adex-protocol#validator-stack-platform).
+# Ambire AdEx Validator Stack [![CI](https://github.com/AdExNetwork/adex-validator-stack-rust/workflows/Continuous%20Integration/badge.svg)](https://github.com/AdExNetwork/adex-validator-stack-rust/actions)
 
 Components:
 
 * [Sentry](#sentry)
 * [Validator worker](#validator-worker)
-* Adapter - Ethereum & Dummy (for testing) Adapters
-* AdView manager
+* [Adapter](./adapter/README.md) - Ethereum & Dummy (for testing) Adapters
+* [AdView manager](./adview-manager/README.md)
 
 ## Local & Testing setup
 
 Requirements:
 
 - Rust
-
-  Check the [`rust-toolchain`](./rust-toolchain) file for specific version of rust.
+  - We target the `stable` version of the Rust compiler.
   - [`cargo-make`](https://github.com/sagiegurari/cargo-make)
-- Docker
-- Node 14 (LTS) & npm
-    Used for running a local Ethereum node with `ganache-cli` for automated tests in the `Ethereum Adapter` ([adapter/src/ethereum.rs](./adapter/src/ethereum.rs), also check [scripts/ethereum.sh](./scripts/ethereum.sh))
+- Docker & Docker-compose
 
 #### Linux
 
@@ -30,17 +23,25 @@ Requirements:
 
 ## Sentry
 
-`Sentry` is the REST API that the [`Validator worker`](#validator-worker) uses for storing and retrieving information.
-We need two services to be able to run `Sentry`: `Postgres` and `Redis`.
+`Sentry` is the REST API that the [`Validator worker`](#running-the-validator-worker)
+uses for storing and retrieving information.
+
+Two services are needed to run `Sentry`: `Postgres` and `Redis`.
+
+The easiest way to run these services locally is by using the provided `docker-compose` file:
 
-### Running Postgres
+`docker-compose -f ../docker-compose.harness.yml up -d adex-redis adex-postgres`
+
+If you want to run them manually without `docker-compose`:
+
+#### Running Postgres
 
 `docker run --rm --name adex-validator-postgres -e POSTGRES_PASSWORD=postgres -d -p 5432:5432 -v $HOME/docker/volumes/postgres:/var/lib/postgresql/data postgres`
 
 - `$HOME/docker/volumes/postgres` - your local storage for postgres (persist the data when we remove the container)
-- `POSTGRES_PASSWORD=postgres` - the password of `postgres` user
+- `POSTGRES_PASSWORD=postgres` - the password of the default `postgres` user
 
-**NOTE:** Additionally you must setup 2 databases - `sentry_leader` & `sentry_follower` in order for the provided examples bellow to work.
+**NOTE:** Additionally you must setup 2 databases - `sentry_leader` & `sentry_follower` in order for the provided examples below to work. Postgres comes with an environment variable `POSTGRES_DB` that you can use to change the default `postgres` database, but there is currently no way to create multiple using the official `postgres` image.
 
 ### Running Redis
 
@@ -56,47 +57,60 @@ cargo run -p sentry -- --help
 
 Starting the Sentry API in will always run migrations, this will make sure the database is always up to date with the latest migrations, before starting and exposing the web server.
 
-In `development` ( [`ENV` environment variable](#environment-variables) ) it will seed the database as well.
+By default, we use the `development` environment ( [`ENV` environment variable](#environment-variables) ) as it will also seed the database.
 
-#### Using the `Ethereum Adapter`
+#### Using the `Ethereum` adapter
 
-The password for the Keystore file can be set using the [environment variable `KEYSTORE_PWD`](#adapter).
+The password for the keystore file can be set using the [`KEYSTORE_PWD` environment variable](#adapter).
+These examples use the Leader and Follower addresses for testing locally with
+`ganache` and the production configuration of the validator.
 
-- Leader
-    ```bash
-    POSTGRES_DB="sentry_leader" PORT=8005 KEYSTORE_PWD=adexvalidator cargo run -p sentry -- \
-        --adapter ethereum \
-        --keystoreFile ./adapter/resources/keystore.json \
-        ./docs/config/dev.toml
-    ```
+##### Leader (`0x80690751969B234697e9059e04ed72195c3507fa`)
 
-- Follower
-    ```bash
-    POSTGRES_DB="sentry_follower" PORT=8006 KEYSTORE_PWD=adexvalidator cargo run -p sentry -- \
-        --adapter ethereum \
-        --keystoreFile ./adapter/resources/keystore.json
-        ./docs/config/dev.toml
-    ```
+Sentry API will be accessible at `localhost:8005`
 
-#### Using the `Dummy Adapter`
+```bash
+IP_ADDR=127.0.0.1 REDIS_URL="redis://127.0.0.1:6379/1" \
+POSTGRES_DB="sentry_leader" PORT=8005 KEYSTORE_PWD=ganache0 \
+cargo run -p sentry -- \
+    --adapter ethereum \
+    --keystoreFile ./adapter/tests/resources/0x80690751969B234697e9059e04ed72195c3507fa_keystore.json \
+    ./docs/config/prod.toml
+```
+
+##### Follower (`0xf3f583AEC5f7C030722Fe992A5688557e1B86ef7`)
+
+Sentry API will be accessible at `localhost:8006`
+
+```bash
+IP_ADDR=127.0.0.1 REDIS_URL="redis://127.0.0.1:6379/2" \
+POSTGRES_DB="sentry_follower" PORT=8006 KEYSTORE_PWD=ganache1 cargo run -p sentry -- \
+    --adapter ethereum \
+    --keystoreFile ./adapter/test/resources/0xf3f583AEC5f7C030722Fe992A5688557e1B86ef7_keystore.json
+    ./docs/config/prod.toml
+```
+
+#### Using the `Dummy` adapter
 
 **Dummy** identities:
 
-- Leader: `ce07CbB7e054514D590a0262C93070D838bFBA2e`
+##### Leader (`0x80690751969B234697e9059e04ed72195c3507fa`)
 
 ```bash
-    POSTGRES_DB="sentry_leader" PORT=8005 cargo run -p sentry -- \
-        --adapter dummy \
-        --dummyIdentity ce07CbB7e054514D590a0262C93070D838bFBA2e \
-        ./docs/config/dev.toml
+IP_ADDR=127.0.0.1 REDIS_URL="redis://127.0.0.1:6379/1" \
+POSTGRES_DB="sentry_leader" PORT=8005 cargo run -p sentry -- \
+    --adapter dummy \
+    --dummyIdentity 80690751969B234697e9059e04ed72195c3507fa \
+    ./docs/config/prod.toml
 ```
-- Follower: `c91763d7f14ac5c5ddfbcd012e0d2a61ab9bded3`
+##### Follower (`0xf3f583AEC5f7C030722Fe992A5688557e1B86ef7`)
 
 ```bash
-    POSTGRES_DB="sentry_follower" PORT=8006 cargo run -p sentry -- \
-        --adapter dummy \
-        --dummyIdentity c91763d7f14ac5c5ddfbcd012e0d2a61ab9bded3 \
-        ./docs/config/dev.toml
+IP_ADDR=127.0.0.1 REDIS_URL="redis://127.0.0.1:6379/2" \
+POSTGRES_DB="sentry_follower" PORT=8006 cargo run -p sentry -- \
+    --adapter dummy \
+    --dummyIdentity f3f583AEC5f7C030722Fe992A5688557e1B86ef7 \
+    ./docs/config/prod.toml
 ```
 
 For full list, check out [primitives/src/util/tests/prep_db.rs#L29-L43](./primitives/src/util/tests/prep_db.rs#L29-L43)
@@ -106,114 +120,119 @@ For full list, check out [primitives/src/util/tests/prep_db.rs#L29-L43](./primit
 - `ENV` - `production` or `development`; *default*: `development` - passing this env. variable will use the default configuration paths - [`docs/config/dev.toml`](./docs/config/dev.toml) (for `development`) or [`docs/config/prod.toml`](./docs/config/prod.toml) (for `production`). Otherwise you can pass your own configuration file path to the binary (check `cargo run -p sentry --help` for more information). In `development` it will make sure Sentry to seed the database.
 - `PORT` - *default*: `8005` - The local port that Sentry API will be accessible at
 - `IP_ADDR` - *default*: `0.0.0.0` - the IP address that the API should be listening to
-- `ANALYTICS_RECORDER` - accepts any non-zero value - whether or not to start the `Analytics recorder` that will track analytics stats for payout events (`IMPRESSION` & `CLICK`)
+
 ##### Adapter
-- `KEYSTORE_PWD` - Password for the `Keystore file`, only available when using `Ethereum Adapter` (`--adapter ethereum`)
+- `KEYSTORE_PWD` - Password for the `Keystore file`, only available when using `Ethereum` adapter (`--adapter ethereum`)
 
 ##### Redis
+
 - `REDIS_URL` - *default*: `redis://127.0.0.1:6379`
 
 ##### Postgres
+
 - `POSTGRES_HOST` - *default*: `localhost`
 - `POSTGRES_USER` - *default*: `postgres`
 - `POSTGRES_PASSWORD` - *default*: `postgres`
 - `POSTGRES_DB` - *default*: `user` name - Database name in Postgres to be used for this instance
 - `POSTGRES_PORT` - *default*: `5432`
 
-#####
 
-### Running the Validator Worker
+### Validator worker
 
 For a full list of all available CLI options on the Validator worker run `--help`:
 
 ```bash
 cargo run -p validator_worker -- --help
 ```
 
-#### Using the `Ethereum Adapter`
-TODO: Update Keystore file and Keystore password for Leader/Follower as they are using the same at the moment.
-
+#### Using the `Ethereum` adapter
 The password for the Keystore file can be set using the environment variable `KEYSTORE_PWD`.
 
-- Leader
+##### Validator Leader (`0x80690751969B234697e9059e04ed72195c3507fa`)
     Assuming you have [Sentry API running](#running-sentry-rest-api) for the **Leader** on port `8005`:
 
-    ```bash
-    KEYSTORE_PWD=adexvalidator cargo run -p validator_worker -- \
-        --adapter ethereum \
-        --keystoreFile ./adapter/resources/keystore.json \
-        --sentryUrl http://127.0.0.1:8005 \
-        ./docs/config/dev.toml
-    ```
+```bash
+KEYSTORE_PWD=ganache0 cargo run -p validator_worker -- \
+    --adapter ethereum \
+    --keystoreFile ./adapter/test/resources/0x80690751969B234697e9059e04ed72195c3507fa_keystore.json \
+    --sentryUrl http://127.0.0.1:8005 \
+    ./docs/config/prod.toml
+```
 
-- Follower
+##### Validator Follower
 
-    Assuming you have [Sentry API running](#running-sentry-rest-api) for the **Follower** on port `8006`:
+Assuming you have [Sentry API running](#running-sentry-rest-api) for the **Follower** on port `8006`:
 
-    ```bash
-    KEYSTORE_PWD=adexvalidator cargo run -p validator_worker -- \
-        --adapter ethereum \
-        --keystoreFile ./adapter/resources/keystore.json \
-        --sentryUrl http://127.0.0.1:8006 \
-        ./docs/config/dev.toml
-    ```
+```bash
+KEYSTORE_PWD=ganache1 cargo run -p validator_worker -- \
+    --adapter ethereum \
+    --keystoreFile ./adapter/test/resources/0xf3f583AEC5f7C030722Fe992A5688557e1B86ef7_keystore.json \
+    --sentryUrl http://127.0.0.1:8006 \
+    ./docs/config/prod.toml
+```
 
-#### Using the `Dummy Adapter`
-- Leader: `ce07CbB7e054514D590a0262C93070D838bFBA2e`
+#### Using the `Dummy` adapter
 
-    Assuming you have [Sentry API running](#running-sentry-rest-api) for the **Leader** on port `8005`:
+##### Validator Leader (`0x80690751969B234697e9059e04ed72195c3507fa`)
 
-    ```bash
-    cargo run -p validator_worker -- \
-        --adapter dummy \
-        --dummyIdentity ce07CbB7e054514D590a0262C93070D838bFBA2e \
-        --sentryUrl http://127.0.0.1:8005 \
-        ./docs/config/dev.toml
-    ```
+Assuming you have [Sentry API running](#running-sentry-rest-api) for the **Leader** on port `8005`:
 
-- Follower: `c91763d7f14ac5c5ddfbcd012e0d2a61ab9bded3`
+```bash
+cargo run -p validator_worker -- \
+    --adapter dummy \
+    --dummyIdentity 0x80690751969B234697e9059e04ed72195c3507fa \
+    --sentryUrl http://127.0.0.1:8005 \
+    ./docs/config/prod.toml
+```
 
-    Assuming you have [Sentry API running](#running-sentry-rest-api) for the **Follower** on port `8006`:
+##### Follower: `0xf3f583AEC5f7C030722Fe992A5688557e1B86ef7`
 
-    ```bash
-    cargo run -p validator_worker -- \
-        --adapter dummy \
-        --dummyIdentity c91763d7f14ac5c5ddfbcd012e0d2a61ab9bded3 \
-        --sentryUrl http://127.0.0.1:8006 \
-        ./docs/config/dev.toml
-    ```
+Assuming you have [Sentry API running](#running-sentry-rest-api) for the **Follower** on port `8006`:
+
+```bash
+cargo run -p validator_worker -- \
+    --adapter dummy \
+    --dummyIdentity 0xf3f583AEC5f7C030722Fe992A5688557e1B86ef7 \
+    --sentryUrl http://127.0.0.1:8006 \
+    ./docs/config/prod.toml
+```
 
 #### Environment variables
 
-- `ENV`: `production` or `development` ( *default* ) - passing this env. variable will use the default configuration paths - [`docs/config/dev.toml`](./docs/config/dev.toml) (for `development`) or [`docs/config/prod.toml`](./docs/config/prod.toml) (for `production`). Otherwise you can pass your own configuration file path to the binary (check `cargo run -p sentry --help` for more information). In `development` it will make sure Sentry to seed the database.
+- `ENV` - `production` or `development` - *default*: `development` - passing this env. variable will use the default configuration paths - [`docs/config/dev.toml`](./docs/config/dev.toml) (for `development`) or [`docs/config/prod.toml`](./docs/config/prod.toml) (for `production`). Otherwise you can pass your own configuration file path to the binary (check `cargo run -p sentry --help` for more information). In `development` it will make sure that Sentry seeds the database.
 - `PORT` - The local port that Sentry API will accessible at
 
 ##### Adapter
+
 - `KEYSTORE_PWD` - Password for the `Keystore file`, only available when using `Ethereum Adapter` (`--adapter ethereum`)
 
 ## Development environment
 
-We use [`cargo-make`](https://github.com/sagiegurari/cargo-make#overview) for running automated checks (tests, builds, formatting, code linting, etc.) and building the project locally
-as well as on our Continuous Integration (CI). For a complete list of out-of-the-box commands you can check
-[Makefile.stable.toml](https://github.com/sagiegurari/cargo-make/blob/master/src/lib/Makefile.stable.toml).
+We use [`cargo-make`][cargo-make overview] for running automated checks
+(tests, builds, formatting, code linting, etc.) and building the project locally
+as well as on our Continuous Integration (CI).
+
+For a complete list of out-of-the-box commands you can check out the [`Predefined Makefiles`](https://github.com/sagiegurari/cargo-make#usage-predefined-makefiles)
+while locally defined commands can be found in the `Makefiles.toml` in each crate directory.
 
 ### Local development
 
-Locally it's enough to ensure that the default development command is executing successfully:
+It's enough to ensure that the default development command is executing successfully:
 
 ```bash
 cargo make
 ```
 
-It will run `rustfmt` for you as well as `clippy` (it will fail on warnings) and it will run all the tests thanks to `cargo` (doc tests, unit tests, integration tests, etc.).
+It will format your code using `rustfmt` and will perform `clippy` checks (it will fail on warnings).
+Thanks to `cargo` it will run all the tests (doc tests, unit tests, integration tests, etc.).
+
+Using the provided `docker-compose.harness.yml` setup [`cargo-make`][cargo-make overview] will run
+all the required services for the specific crate/application before executing the tests.
 
-This will also run the [Automated tests](#automated-tests), so you must have `Redis` & `Postgres` running.
+[cargo-make overview]: https://github.com/sagiegurari/cargo-make#overview
 
-#### Automated tests
 
-This requires [`cargo-make`](https://github.com/sagiegurari/cargo-make#overview) and since we have integration tests that require `Redis` ([see `Running Redis`](#running-redis)) & `Postgres` (see [`Running Postgres`](#running-postgres)), you need to be running those in order to run the automated tests:
 
-`cargo make test`
+### License
 
-You can relate to the [`Makefile.stable.toml`](https://github.com/sagiegurari/cargo-make/blob/master/src/lib/Makefile.stable.toml)
-for more commands and cargo-make as a whole.
+This project is licensed under the [AGPL-3.0 license](./LICENSE)

adapter/Makefile.toml

@@ -17,6 +17,7 @@ dependencies = [
 args = ["test", "--release", "--all-features"]
 
 [tasks.ganache-up]
+# `--renew-anon-volumes` is required to make sure that the snapshot used for the `ganache` instances is reloaded.
 script = '''
 docker-compose -f ../docker-compose.harness.yml up --renew-anon-volumes -d ganache-1 ganache-1337 \
 && sleep 10

adview-manager/README.md

@@ -1,6 +1,7 @@
 # AdView Manager
 
-Gives you the ability to generate the code for showing ads on a publisher website.
+Gives you the ability to generate the code required for showing ads
+on a website (publisher).
 
 ### Testing
 

docker-compose.harness.yml

@@ -12,6 +12,9 @@ services:
       POSTGRES_HOST: 'localhost'
       POSTGRES_USER: 'postgres'
       POSTGRES_PASSWORD: 'postgres'
+      # harness_* databases are used by the `test_harness` crate for testing
+      # `sentry_leader` is the default database used by `sentry` for running tests
+      # `primitives` is used for running tests in the `primitives` crate
       POSTGRES_MULTIPLE_DATABASES: harness_leader,harness_follower,sentry_leader,primitives
     networks:
       - adex-external

sentry/Migrant.toml

@@ -4,7 +4,7 @@ database_type = "postgres"
 # Required database info
 database_name = "sentry_leader"
 database_user = "postgres"
-database_password = "docker"
+database_password = "postgres"
 
 # Configurable database info
 # default "localhost"