Added Postgres as an additional persistency layer (as well as some cosmetic changes to the app UI)

Author: mreferre

README.md

@@ -6,17 +6,17 @@ Yelb is an app I wrote from scratch (leveraging existing frameworks and tools su
 I have built it for a couple of reasons: 
 
 - while there are dozens of demo/test apps out there, I think that having an app that you know 100% end-to-end and that you know inside out is something useful (if nothing, you will learn/experience one or more programming languages) 
-- building a set of assets to deploy the application on multiple platforms (e.g. Docker, Kubernetes, Rancher, CloudFoundry, OpenShift, ECS, etc etc). Owning the application itself makes this task easier, more useful and more interesting. 
+- building a set of assets to deploy a "real" application on multiple platforms (e.g. Docker, Kubernetes, Rancher, CloudFoundry, OpenShift, ECS, Serverless etc etc). Owning the application itself makes this task easier, more useful and more interesting. Also, using target platforms specific examples isn't quite like the challenge of having to think about how to adapt an existing application to be able to deploy it on a specific target platform.   
 
-As far as reason #1 goes, I found very interesting dealing with the challenges (at the edge of dev and ops duties) of starting and coordinating the application modules in specific environments (development, test, production). This is often achieved in specific programming environments leveraging system variables (e.g. RACK_ENV for ruby, other mechanisms for Angular2, etc). You will see some of these configurations in actions in the `localdevelopment` and `localtest` instructions (inside the `deployments` folder). Building an application from scratch and packaging it using Docker while honoring this flexibility is a very interesting (learning) exercise.   
+As far as reason #1 goes, I found very interesting dealing with the challenges (at the edge of dev and ops duties) of starting and coordinating the application modules in specific environments (development, test, production). This is often achieved in specific programming environments leveraging system variables (e.g. RACK_ENV for ruby, other mechanisms for Angular2, etc). You will see some of these configurations in actions in the `localdevelopment` and `localtest` instructions (inside the `deployments` folder). Building an application from scratch and packaging it using Docker while honoring this flexibility is a very interesting (learning) exercise.
 
 #### Yelb architecture
 
 The current architecture layout is pretty straightforward. 
 
 There is a front-end component called `yelb-ui` that fullfills a couple of roles (they could even be split if need be). The first role is to host the Angular 2 application (i.e. the UI of the Yelb application). When the browser connects to this layer it downloads the `Javascript` code that builds the UI itself. Subsequent requests and calls to other application components are proxied via the `nginx` service running on `yelb-ui`. 
 
-At the time of this writing the only application component that exists is `yelb-appserver`. This is a Sinatra application that basically read and write to a third component which is the `redis-server` backend database (the third component of the Yelb application). At this point Redis is used more as a backend data base than as a memory cache. In the future it may be possible to insert a true backend system (e.g. Mongo) to store "transactional" data and keep using Redis for storing less critical information. 
+At the time of this writing the only application component that exists is `yelb-appserver`. This is a Sinatra application that basically read and write to a cache server (`redis-server`) as well as a Postgres backend database (`yelb-db`). Redis is used to store the number of page views whereas Postgres is used to persist the votes. Note that the `yelb-db` container image is nothing more than Postgres (10) customized to create the database schema the application relies on.   
 
 The picture below shows the architecture of Yelb:
 
@@ -30,16 +30,18 @@ This is how the UI looks like at this point:
 
 ![yelb-ui](yelb-ui.png)
 
-The nature of the app allows people to experiment with the statless `yelb-ui` and `yelb-appserver` components (think scaling out, blue/green depoyments, etc.) as well as experiment with the stateful `redis-server` backend component (think leveraging persistent volumes, etc.). 
+The nature of the app allows people to experiment with the statless `yelb-ui` and `yelb-appserver` components (think scaling out, blue/green depoyments, etc.) as well as experiment with the stateful `redis-server` and `yelb-db` backend components. 
 
 #### How can I use it?
 
 If you intend to contribute, fork or understand anyway how the single components work I suggest you dig into each of them looking at the code (and the various dockerfiles) to learn how they work and how they are packaged. In addition to that you should look into the `deployments` folder how to start the application in development mode and test mode.  
 
-If you intend to use Yelb as a generic application with the ultimate goal of learning and playing with container platforms, then you should probably go straight into the `deployments` folder and specifically in the `platformdeployment` folder. This is where all the various configuration files for the various platforms exist (or will exist). This folder will ideally be a continuous work in progress. 
+If you intend to use Yelb as a generic application with the ultimate goal of learning and playing with multiple platforms (from cloud instances, to containers through potentially serverless), then you should probably go straight into the `deployments` folder and specifically in the `platformdeployment` folder. This is where all the various configuration files for the various platforms exist (or will exist). This folder will ideally be a continuous work in progress. 
 
 #### Known issues and limitations
 
 - There is a subtle issue when you browse the app for the first time where the browser console shows errors. This is likely due to problems with variables not initialized properly. This needs further investigation. The app works just fine and there is no evidence of the problem unless you inspect the page and go to the console outputs.   
-- There is a ton of optimization that could be achieved both in the application code as well as in the packaging constructs (e.g. Dockerfiles). The package in its current form works but it is far from being considered optmized. 
-- While the intent was to build an application whose layers could scale out (and scale independently) in a classic microservices scenario, very little testing has gone into multi-container deployment scenarios so far. Most of the tests have been done with one (1) container per layer.
+- There is a ton of optimization that could be achieved both in the application code as well as in the packaging constructs (e.g. Dockerfiles). The package in its current form works but it is far from being considered optmized.
+- Idempotency may be a problem. While I am trying to explicitly declare container tags and modules versions moving forward, there is a chance that if you try to build your own containers from the Dockerfile provided and/or install the project on a cloud instance, the output may be diverging from one setup to another. Generic `apt-get update`, `npm install` `gem install sinatra` instructions may lead to different versions of code and modules being pulled at build and setup times.
+- The cache instance is still named `redis-server` (this will be renamed `yelb-cache` in the future for consistency)
+- While the intent was to build an application whose layers could scale out (and scale independently) in a classic microservices scenario, very little testing has gone into scale out deployment scenarios so far. Most of the tests have been done with one (1) instance of service per layer.

deployments/localdevelopment/setupdevenv.sh

@@ -1,9 +1,10 @@
 
 #!/bin/bash
 
-# workstation requirements: git, Docker, Ruby (with proper libraries), angular@CLI 
+# workstation requirements: git, Docker, Ruby (with proper libraries/gems), angular@CLI 
 
-docker run --name redis -p 6379:6379 -d redis 
+docker run --name redis -p 6379:6379 -d redis:4.0.2
+docker run --name postgres -p 5432:5432 -d mreferre/yelb-db:0.2
 
 cd ./yelb/yelb-appserver
 export RACK_ENV=development 

deployments/localtest/README.md

@@ -6,9 +6,10 @@ Move to the directory where you want to work and clone the repo:
 
 ```
 docker network create yelb-network 
-docker run --name redis-server -p 6379:6379 --network=yelb-network -d redis
-docker run --name yelb-appserver --network=yelb-network -d -p 4567:4567 -e RACK_ENV=test mreferre/yelb-appserver
-docker run --name yelb-ui --network=yelb-network -d -p 8080:80 -e UI_ENV=test mreferre/yelb-ui
+docker run --name redis-server -p 6379:6379 --network=yelb-network -d redis:4.0.2
+docker run --name yelb-db -p 5432:5432 --network=yelb-network -d mreferre/yelb-db:0.2
+docker run --name yelb-appserver --network=yelb-network -d -p 4567:4567 -e RACK_ENV=test mreferre/yelb-appserver:0.2
+docker run --name yelb-ui --network=yelb-network -d -p 8080:80 -e UI_ENV=test mreferre/yelb-ui:0.2
 ```
 You should now be able to see the application running by connecting your browser to: http://localhost:8080
 

deployments/localtest/docker-compose.yaml

@@ -1,8 +1,8 @@
-#use this with `docker-compose up` against a generic Docker end-point (e.g. standalone Docker, VIC)
+#use this with `docker-compose up` against a generic Docker end-point (e.g. standalone Docker)
 version: "2.1"
 services:
   yelb-ui:
-    image: mreferre/yelb-ui
+    image: mreferre/yelb-ui:0.2
     depends_on:
       - yelb-appserver
     ports:
@@ -11,15 +11,21 @@ services:
       - UI_ENV=test # dev | test | prod 
 
   yelb-appserver:
-    image: mreferre/yelb-appserver
+    image: mreferre/yelb-appserver:0.2
     depends_on:
       - redis-server
+      - yelb-db
     ports:
       - 4567:4567
     environment:
       - RACK_ENV=test # development | test | production 
 
   redis-server:
-    image: redis
+    image: redis:4.0.2
     ports:
       - 6379:6379
+
+  yelb-db:
+    image: mreferre/yelb-db:0.2
+    ports:
+      - 5432:5432

deployments/platformdeployment/Docker/README.md

@@ -1,5 +1,5 @@
 These configurations had been lightly tested against Docker for Mac (configured as a single node Swarm cluster).
 
-These two scripts should work against a simple docker host (which includes support for a VMware VCH or a `Swarm legacy` cluster) as well as a `Swarm mode` cluster.
+These two scripts should work against a simple docker host (which includes support for `Swarm legacy` clusters) as well as a `Swarm mode` cluster.
 
 This is a WIP.

deployments/platformdeployment/Docker/docker-compose.yaml

@@ -1,8 +1,8 @@
-# use this with `docker-compose up` against a generic Docker end-point (e.g. standalone Docker, VIC)
+# use this with `docker-compose up` against a generic Docker end-point (e.g. standalone Docker)
 version: "2.1"
 services:
   yelb-ui:
-    image: mreferre/yelb-ui
+    image: mreferre/yelb-ui:0.2
     depends_on:
       - yelb-appserver
     ports:
@@ -11,25 +11,36 @@ services:
       - yelb-network
 
   yelb-appserver:
-    image: mreferre/yelb-appserver
+    image: mreferre/yelb-appserver:0.2
     depends_on:
       - redis-server
+      - yelb-db
     networks:
       - yelb-network
 
   redis-server:
-    image: redis
+    image: redis:4.0.2
     networks:
       - yelb-network
     # uncomment the following lines if you want to persist redis data across deployments
     #volumes:
-    #  - data:/data
+    # - redisdata:/data
+
+  yelb-db:
+    image: mreferre/yelb-db:0.2
+    networks:
+      - yelb-network
+    # uncomment the following lines if you want to persist postgres data across deployments
+    #volumes:
+    #  - postgresqldata:/var/lib/postgresql/data
 
 networks:
   yelb-network:
-    driver: bridge # a user defined bridge is required; the default bridge network doesn't support name resolution   
+    driver: bridge # a user defined bridge is required; the default bridge network doesn't support name resolution
 
-# uncomment the following lines if you want to persist redis data across deployments
+# uncomment the following lines if you want to persist redis/postgres data across deployments
 #volumes:
-#   data:
-#      driver: local # you can pick another driver depending on the platform you are deploying onto
+#   redisdata:
+#      driver: local # you can pick another driver depending on the platform you are deploying onto
+#   postgresqldata:
+#      driver: local # you can pick another driver depending on the platform you are deploying onto

deployments/platformdeployment/Docker/stack-deploy.yaml

@@ -2,7 +2,7 @@
 version: "3"
 services:
   yelb-ui:
-    image: mreferre/yelb-ui
+    image: mreferre/yelb-ui:0.2
     depends_on:
       - yelb-appserver
     ports:
@@ -15,9 +15,10 @@ services:
         condition: on-failure
 
   yelb-appserver:
-    image: mreferre/yelb-appserver
+    image: mreferre/yelb-appserver:0.2
     depends_on:
       - redis-server
+      - yelb-db
     networks:
       - yelb-network
     deploy:
@@ -26,12 +27,24 @@ services:
         condition: on-failure
 
   redis-server:
-    image: redis
+    image: redis:4.0.2
     networks:
       - yelb-network
     # uncomment the following lines if you want to persist redis data across deployments
     #volumes:
-    #  - data:/data
+    #  - redisdata:/data
+    deploy:
+      replicas: 1
+      restart_policy:
+        condition: on-failure
+
+  yelb-db:
+    image: mreferre/yelb-db:0.2
+    networks:
+      - yelb-network
+    # uncomment the following lines if you want to persist postgres data across deployments
+    #volumes:
+    #  - postgresqldata:/var/lib/postgresql/data
     deploy:
       replicas: 1
       restart_policy:
@@ -41,7 +54,10 @@ networks:
   yelb-network:
     driver: overlay # a user defined bridge is required; the default bridge network doesn't support name resolution
 
-# uncomment the following lines if you want to persist redis data across deployments
+# uncomment the following lines if you want to persist redis/postgres data across deployments
 #volumes:
-#   data:
-#      driver: local # you can pick another driver depending on the platform you are deploying onto   
+#   redisdata:
+#      driver: local # you can pick another driver depending on the platform you are deploying onto   
+#   postgresqldata:
+#      driver: local # you can pick another driver depending on the platform you are deploying onto
+

deployments/platformdeployment/Kubernetes/cnawebapp-minikube-hostport.yaml

@@ -6,14 +6,29 @@ metadata:
   name: redis-server
   labels:
     app: redis-server
-    tier: backend
+    tier: cache
 spec:
   type: ClusterIP
   ports:
   - port: 6379
   selector:
     app: redis-server
-    tier: backend
+    tier: cache
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: yelb-db
+  labels:
+    app: yelb-db
+    tier: backenddb
+spec:
+  type: ClusterIP
+  ports:
+  - port: 5432
+  selector:
+    app: yelb-db
+    tier: backenddb
 ---
 apiVersion: v1
 kind: Service
@@ -45,7 +60,7 @@ spec:
     spec:
       containers:
       - name: yelb-ui
-        image: mreferre/yelb-ui
+        image: mreferre/yelb-ui:0.2
         ports:
         - containerPort: 80
           hostPort: 32777 # depending on the minikube driver you use you may be able to even use 80. But something above 32000 seems safer 
@@ -60,16 +75,34 @@ spec:
     metadata:
       labels:
         app: redis-server
-        tier: backend
+        tier: cache
     spec:
       containers:
       - name: redis-server
-        image: redis
+        image: redis:4.0.2
         ports:
         - containerPort: 6379
 ---
 apiVersion: extensions/v1beta1
 kind: Deployment
+metadata:
+  name: yelb-db
+spec:
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        app: yelb-db
+        tier: backenddb
+    spec:
+      containers:
+      - name: yelb-db
+        image: mreferre/yelb-db:0.2
+        ports:
+        - containerPort: 5432
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
 metadata:
   name: yelb-appserver
 spec:
@@ -82,7 +115,7 @@ spec:
     spec:
       containers:
       - name: yelb-appserver
-        image: mreferre/yelb-appserver
+        image: mreferre/yelb-appserver:0.2
         ports:
         - containerPort: 4567