Templated Redis Connection String Formation. Replaced Etcd references with Redis.

Author: arms11

.circleci/config.yml

@@ -50,7 +50,7 @@ jobs:
   # Spin up minikube K8s cluster and run Helm chart & e2e tests on it
   helm-e2e:
     # 'large' 4 vCPUs & 15GB RAM CircleCI machine executor
-    # required to deploy heavy 'stackstorm-ha' Helm release with RabbitMQ, MongoDB, etcd clusters and 25+ st2 Pods.
+    # required to deploy heavy 'stackstorm-ha' Helm release with RabbitMQ, MongoDB, Redis clusters and 25+ st2 Pods.
     # https://circleci.com/docs/2.0/configuration-reference/#machine-executor-linux
     resource_class: large
     machine:

Chart.yaml

@@ -40,10 +40,6 @@ dependencies:
     version: 4.0.0
     repository: https://charts.bitnami.com/bitnami
     condition: external-dns.enabled
-  - name: etcd
-    version: 5.1.0
-    repository: https://charts.bitnami.com/bitnami
-    condition: etcd.enabled
   - name: redis
     version: 12.3.2
     repository: https://charts.bitnami.com/bitnami

README.md

@@ -4,7 +4,7 @@
 K8s Helm Chart for running StackStorm cluster in HA mode.
 
 It will install 2 replicas for each component of StackStorm microservices for redundancy, as well as backends like
-RabbitMQ HA, MongoDB HA Replicaset and etcd cluster that st2 replies on for MQ, DB and distributed coordination respectively.
+RabbitMQ HA, MongoDB HA Replicaset and Redis cluster that st2 replies on for MQ, DB and distributed coordination respectively.
 
 It's more than welcome to fine-tune each component settings to fit specific availability/scalability demands.
 
@@ -116,7 +116,7 @@ All the workflow engine processes will share the load and pick up more work if o
 Multiple st2notifier processes can run in active-active mode, using connections to RabbitMQ and MongoDB and generating triggers based on
 action execution completion as well as doing action rescheduling.
 In an HA deployment there must be a minimum of `2` replicas of st2notifier running, requiring a coordination backend,
-which in our case is `etcd`.
+which in our case is `Redis`.
 
 ### [st2sensorcontainer](https://docs.stackstorm.com/reference/ha.html#st2sensorcontainer)
 st2sensorcontainer manages StackStorm sensors: It starts, stops and restarts them as subprocesses.
@@ -143,7 +143,7 @@ st2:
 ### [st2actionrunner](https://docs.stackstorm.com/reference/ha.html#st2actionrunner)
 Stackstorm workers that actually execute actions.
 `5` replicas for K8s Deployment are configured by default to increase StackStorm ability to execute actions without excessive queuing.
-Relies on `etcd` for coordination. This is likely the first thing to lift if you have a lot of actions
+Relies on `redis` for coordination. This is likely the first thing to lift if you have a lot of actions
 to execute per time period in your StackStorm cluster.
 
 ### [st2garbagecollector](https://docs.stackstorm.com/reference/ha.html#st2garbagecollector)
@@ -174,9 +174,9 @@ Helm chart repository, - all settings could be overridden via `values.yaml`.
 
 The deployment of RabbitMQ to the k8s cluster can be disabled by setting the rabbitmq-ha.enabled key in values.yaml to false.  *Note: Stackstorm relies heavily on connections to a RabbitMQ instance.  If the in-cluster deployment of RabbitMQ is disabled, a connection to an external instance of RabbitMQ must be configured.  The st2.config key in values.yaml provides a way to configure stackstorm.  See [Configure RabbitMQ](https://docs.stackstorm.com/install/config/config.html#configure-rabbitmq) for configuration details.*
 
-### [etcd](https://docs.stackstorm.com/latest/reference/ha.html#zookeeper-redis)
-StackStorm employs etcd as a distributed coordination backend, required for st2 cluster components to work properly in HA scenario.
-`3` node Raft cluster is deployed via external bitnami Helm chart dependency [etcd](https://github.com/bitnami/charts/tree/master/bitnami/etcd).
+### [redis](https://docs.stackstorm.com/latest/reference/ha.html#zookeeper-redis)
+StackStorm employs redis sentinel as a distributed coordination backend, required for st2 cluster components to work properly in HA scenario.
+`3` node Redis cluster with Sentinel enabled is deployed via external bitnami Helm chart dependency [redis](https://github.com/bitnami/charts/tree/master/bitnami/redis).
 As any other Helm dependency, it's possible to further configure it for specific scaling needs via `values.yaml`.
 
 ## Install custom st2 packs in the cluster
@@ -220,7 +220,7 @@ Grab all logs for entire StackStorm cluster with dependent services in Helm rele
 kubectl logs -l release=<release-name>
 ```
 
-Grab all logs only for stackstorm backend services, excluding st2web and DB/MQ/etcd:
+Grab all logs only for stackstorm backend services, excluding st2web and DB/MQ/redis:
 ```
 kubectl logs -l release=<release-name>,tier=backend
 ```

templates/_helpers.tpl

@@ -76,3 +76,19 @@ Create the name of the stackstorm-ha service account to use
   {{- end -}}
 {{- end -}}
 {{- end -}}
+
+# Generate list of nodes for Redis with Sentinel connection string, based on number of replicas and service name
+{{- define "redis-nodes" -}}
+{{- if not .Values.redis.sentinel.enabled }}
+{{- fail "value for redis.sentinel.enabled MUST be true" }}
+{{- end }}
+{{- $replicas := (int (index .Values "redis" "cluster" "slaveCount")) }}
+{{- range $index0 := until $replicas -}}
+  {{- if eq $index0 0 -}}
+    {{ $.Release.Name }}-redis-node-{{ $index0 }}.{{ $.Release.Name }}-redis-headless:26379?sentinel=mymaster
+  {{- else -}}
+    &sentinel_fallback={{ $.Release.Name }}-redis-node-{{ $index0 }}.{{ $.Release.Name }}-redis-headless:26379
+  {{- end -}}
+{{- end -}}
+{{- end -}}
+    

templates/configmaps_st2-conf.yaml

@@ -20,13 +20,9 @@ data:
   st2.docker.conf: |
     [auth]
     api_url = http://{{ .Release.Name }}-st2api{{ template "enterpriseSuffix" . }}:9101/
-    {{- if index .Values "etcd" "enabled" }}
-    [coordination]
-    url = etcd://{{ index .Values "etcd" "fullnameOverride" }}:2379
-    {{- end }}
     {{- if index .Values "redis" "enabled" }}
     [coordination]
-    url = redis://{{ .Release.Name }}-redis-node-0.{{ .Release.Name }}-redis-headless:26379?sentinel=mymaster&sentinel_fallback={{ .Release.Name }}-redis-node-1.{{ .Release.Name }}-redis-headless:26379&sentinel_fallback={{ .Release.Name }}-redis-node-2.{{ .Release.Name }}-redis-headless:26379
+    url = redis://{{ template "redis-nodes" $ }}
     {{- end }}
     {{- if index .Values "rabbitmq" "enabled" }}
     [messaging]

values.yaml

@@ -508,20 +508,6 @@ rabbitmq:
   metrics:
     enabled: false
 
-##
-## Etcd HA configuration (3rd party chart dependency)
-##
-## For values.yaml reference:
-## https://github.com/bitnami/charts/tree/master/bitnami/etcd
-##
-etcd:
-  # Change to `false` to disable in-cluster ectd deployment.
-  # Specify your external [coordination] connection parameters under st2.config
-  enabled: false
-  statefulset:
-    replicaCount: 3
-  fullnameOverride: etcd
-
 ##
 ## Redis HA configuration (3rd party chart dependency)
 ##
@@ -532,18 +518,20 @@ redis:
   # Change to `false` to disable in-cluster redis deployment.
   # Specify your external [coordination] connection parameters under st2.config
   enabled: true
+  # By default the cluster is enabled for the subchart.
+  # We just need replica count here to ensure it is HA
   cluster:
-    enabled: true
     slaveCount: 3
   # ## Sentinel settings. Sentinel is enabled for better resiliency.
+  # ## This is highly recommended as per tooz library documentation. 
+  # ## Hence, the chart requires the setting.
   # ## https://docs.openstack.org/tooz/latest/user/drivers.html#redis
   # ## https://github.com/bitnami/charts/tree/master/bitnami/redis#master-slave-with-sentinel
   sentinel:
     enabled: true
   networkPolicy:
     enabled: false
   usePassword: false
-  #password: "PLoT2g3gvD"
   metrics:
     enabled: false