Skip to content

Commit 8179ea5

Browse files
schbetsyschbetsy
committed
README improvements and cleanup
Instructions for gracefully stopping project Move detailed instructions for running dependencies into separate file.
1 parent 9bb589c commit 8179ea5

File tree

2 files changed

+90
-74
lines changed

2 files changed

+90
-74
lines changed

‎Documents/local-dependencies.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
This project depends on several services, which all need to be running in order for the project to operate fully.
2+
3+
Assuming you have Docker-Compose installed, the easiest way to get all of the platform's dependencies up and running with the provided docker-compose dev setup:
4+
5+
`docker-compose -f docker-dev.yml up`
6+
7+
Alternatively, you can start each one individually with the following instructions.
8+
9+
10+
##### Write Journal
11+
12+
* The write side of this system is supported by either a local `leveldb` database or Cassandra. By default, the local `leveldb` is utilized, and some sample data is loaded automatically.
13+
14+
If using `Cassandra` is desired, set the following environment variable:
15+
16+
```shell
17+
export HDMA_IS_DEMO=false
18+
```
19+
20+
The easiest way to run a Cassandra server to support this application for testing is to do it through Docker:
21+
22+
```shell
23+
docker run --name cassandra -p 9042:9042 -p 7000:7000 -p 7199:7199 cassandra:3.10
24+
```
25+
26+
If you want to connect to this server, the following `docker` command will give you access to the Cassandra instance started in the previous step:
27+
28+
```shell
29+
docker run -it --link cassandra:cassandra --rm cassandra cqlsh cassandra
30+
```
31+
32+
Once the `Cassandra` server is running, set the following environment variable to the appropriate Cassandra host (in this example, the default local docker host for a machine running MacOs X):
33+
34+
```shell
35+
export CASSANDRA_CLUSTER_HOSTS=192.168.99.100
36+
```
37+
38+
To load data into `Cassandra`, you can run the following (the Cassandra server needs to be running and correct environment variables configured as per the previous instructions):
39+
40+
```shell
41+
$ sbt
42+
project panel
43+
run <full local path to sample file>
44+
```
45+
A sample file is located in the following folder: `panel/src/main/resources/inst_data_2017_dummy.csv`
46+
47+
##### Read Journal
48+
49+
* In order to support the read side, a local PostgreSQL and Cassandra server are needed. Assuming it runs on the default port, on the same machine as the API, the following environment variable needs to be set:
50+
51+
```shell
52+
export JDBC_URL='jdbc:postgresql://localhost/hmda?user=postgres&password=postgres'
53+
```
54+
55+
where `hmda` is the name of the `PostgreSQL` database, owned by the default user with default password (`postgres`)
56+
57+
**Note: if you are running the backend only through sbt, the database needs to be created manually in advance, see instructions [here](https://www.postgresql.org/docs/9.1/static/manage-ag-createdb.html)**
58+
59+
For Cassandra, the following environment variables need to be set (assuming Cassandra is running on a docker container as described above):
60+
61+
```shell
62+
export CASSANDRA_CLUSTER_HOSTS=192.168.99.100
63+
export CASSANDRA_CLUSTER_PORT=9042
64+
```
65+
66+
##### Apache Zookeeper
67+
68+
* The `HMDA Platform` is a distributed system that is meant to be run as a clustered application in production.
69+
As such, it needs a mechanism for storing configuration information for additional nodes joining the cluster.
70+
`Apache Zookeeper` is used to store this information. To run the project, zookeeper must be running and available in the local network.
71+
An easy way to satisfy this requirement is to launch a docker container with `ZooKeeper`, as follows:
72+
73+
```shell
74+
$ docker run --rm -p 2181:2181 -p 2888:2888 -p 3888:3888 jplock/zookeeper
75+
```
76+
77+
* Set the environement variables for Zookeper
78+
79+
```shell
80+
export ZOOKEEPER_HOST=192.168.99.100
81+
export ZOOKEEPER_PORT=2181
82+
```

‎README.md

Lines changed: 8 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -84,88 +84,20 @@ Assuming you have Docker-Compose installed, the easiest way to get all of the pl
8484

8585
`docker-compose -f docker-dev.yml up`
8686

87-
Alternatively, you can start each one individually with the following instructions.
87+
When finished, use `docker-compose down` to gracefully stop the running containers.
8888

89-
##### Write Journal
90-
91-
* The write side of this system is supported by either a local `leveldb` database or Cassandra. By default, the local `leveldb` is utilized, and some sample data is loaded automatically.
92-
If using `Cassandra` is desired, the following environment variable needs to be set:
93-
94-
```shell
95-
export HDMA_IS_DEMO=false
96-
```
97-
98-
The easiest way to run a Cassandra server to support this application for testing is to do it through Docker:
99-
100-
```shell
101-
docker run --name cassandra -p 9042:9042 -p 7000:7000 -p 7199:7199 cassandra:3.10
102-
```
103-
104-
If you want to connect to this server, the following `docker` command will give you access to the Cassandra instance started in the previous step:
105-
106-
```shell
107-
docker run -it --link cassandra:cassandra --rm cassandra cqlsh cassandra
108-
```
109-
110-
Once the `Cassandra` server is running, set the following environment variable to the appropriate Cassandra host (in this example, the default local docker host for a machine running MacOs X):
111-
112-
```shell
113-
export CASSANDRA_CLUSTER_HOSTS=192.168.99.100
114-
```
115-
116-
To load data into `Cassandra`, you can run the following (the Cassandra server needs to be running and correct environment variables configured as per the previous instructions):
117-
118-
```shell
119-
$ sbt
120-
project panel
121-
run <full local path to sample file>
122-
```
123-
A sample file is located in the following folder: `panel/src/main/resources/inst_data_2017_dummy.csv`
124-
125-
##### Read Journal
126-
127-
* In order to support the read side, a local PostgreSQL and Cassandra server are needed. Assuming it runs on the default port, on the same machine as the API, the following environment variable needs to be set:
128-
129-
```shell
130-
export JDBC_URL='jdbc:postgresql://localhost/hmda?user=postgres&password=postgres'
131-
```
132-
133-
where `hmda` is the name of the `PostgreSQL` database, owned by the default user with default password (`postgres`)
134-
135-
**Note: if you are running the backend only through sbt, the database needs to be created manually in advance, see instructions [here](https://www.postgresql.org/docs/9.1/static/manage-ag-createdb.html)**
136-
137-
For Cassandra, the following environment variables need to be set (assuming Cassandra is running on a docker container as described above):
138-
139-
```shell
140-
export CASSANDRA_CLUSTER_HOSTS=192.168.99.100
141-
export CASSANDRA_CLUSTER_PORT=9042
142-
```
143-
144-
##### Apache Zookeeper
145-
146-
* The `HMDA Platform` is a distributed system that is meant to be run as a clustered application in production.
147-
As such, it needs a mechanism for storing configuration information for additional nodes joining the cluster.
148-
`Apache Zookeeper` is used to store this information. To run the project, zookeeper must be running and available in the local network.
149-
An easy way to satisfy this requirement is to launch a docker container with `ZooKeeper`, as follows:
150-
151-
```shell
152-
$ docker run --rm -p 2181:2181 -p 2888:2888 -p 3888:3888 jplock/zookeeper
153-
```
154-
155-
* Set the environement variables for Zookeper
156-
157-
```shell
158-
export ZOOKEEPER_HOST=192.168.99.100
159-
export ZOOKEEPER_PORT=2181
160-
```
89+
Alternatively, you can start each one individually by following the instructions in the [Local Dependencies](documents/local-depencencies.md) documentation.
16190

16291

16392
#### Running the API
16493

165-
* If you want to use the sample files in this repo for testing the app, run the edits in demo mode by adding the following environment variable. Otherwise, edit S025 will trigger for all files.
94+
* For smoke testing locally, add the following two environment variables:
95+
* `EDITS_DEMO_MODE`: This will allow you to use the sample files in this repo for testing the app. Otherwise, edit S025 will trigger for all files.
96+
* `HMDA_IS_DEMO`: This uses configuration files that allow running the app locally, instead of in a cluster.
16697

16798
```shell
16899
export EDITS_DEMO_MODE=true
100+
export HMDA_IS_DEMO=true
169101
```
170102

171103
* Start `sbt`
@@ -185,6 +117,8 @@ $ sbt
185117

186118
Confirm that the platform is up and running by browsing to http://localhost:8080
187119

120+
When finished, press enter to get the sbt prompt, then stop the project by entering `reStop`.
121+
188122

189123
#### Building the Project
190124

0 commit comments

Comments
 (0)