Merge branch 'master' into ubuntu_20

Author: cognifloyd

.github/workflows/unit-tests.yaml

@@ -0,0 +1,45 @@
+---
+name: Unit Tests
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    branches:
+      - master
+
+jobs:
+  helm-unittest:
+
+    runs-on: ubuntu-latest
+#    strategy:
+#      matrix:
+        # Relevant tools installed by default on ubuntu 20.04:
+        #   - helm 3.8.0
+        #   - jq 1.6
+        #   - kind 0.11.1
+        #   - kubectl 1.23.3
+        #   - minikube 1.25.1
+        #   - python 3.8.10
+        #   - yamllint 1.26.3
+        #   - yq 4.19.1
+        # see: https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu2004-Readme.md
+
+    steps:
+    - uses: actions/checkout@v2
+
+    - name: Install helm-unittest
+      # We should periodically check to see if another fork has taken over maintenance,
+      # as the de-facto "best" fork has changed several times over the years.
+      run: |
+        helm plugin install https://github.com/quintush/helm-unittest
+
+    - name: Install chart dependencies
+      run: |
+        helm dependency update
+
+    - name: Run helm-unittest
+      # by default looks for tests/*_test.yaml
+      run: |
+        helm unittest --color --helm3 -f 'tests/unit/*_test.yaml' .

CHANGELOG.md

@@ -1,9 +1,16 @@
 # Changelog
 
 ## In Development
+* Migrate from `python 3.6` `Ubuntu Bionic` to `python 3.8` `Ubuntu Focal` as a base StackStorm OS (StackStorm/st2-dockerfiles#54)
+
+## v0.90.0
 * Advanced Feature: Make securityContext (on Deployments/Jobs) and podSecurityContext (on Pods) configurable. This allows dropping all capabilities, for example. You can override the securityContext for `st2actionrunner`, `st2sensorcontainer`, and `st2client` if your actions or sensors need, for example, additional capabilites that the rest of StackStorm does not need. (#271) (by @cognifloyd)
 * Prefix template helpers with chart name and format helper comments as template comments. (#272) (by @cognifloyd)
-* Migrate from `python 3.6` `Ubuntu Bionic` to `python 3.8` `Ubuntu Focal` as a base StackStorm OS (StackStorm/st2-dockerfiles#54)
+* New feature: Add `extra_volumes` to all python-based st2 deployments. This can facilitate changing log levels by loading logging conf file(s) from a custom ConfigMap. (#276) (by @cognifloyd)
+* Initialize basic unittest infrastructure using `helm-unittest`. Added tests for labels, custom annotations, SecurityContext, pullSecrets, pullPolicy, Resources, nodeSelector, tolerations, affinity, dnsPolicy, dnsConfig, ServiceAccount attach, postStartScript, both sensor-modes, env, envFrom, st2.packs.images, and st2.packs.volumes. (#284, #288, #292)
+* Allow partitioning sensors using the hash_range strategy instead of one sensor per pod. (#218) (by @cognifloyd)
+* New feature to include possibility for external services in st2api, st2stream and st2auth, setting default value for this services as `ClusterIP` and `hostname: ""`. Also, added new entry for custom_annotations_test.yaml and created new unit test services_test.yaml. (by @sandesvitor)
+* Advanced Feature: Add extra Helm hook Jobs. This minimizes the boilerplate required to run stackstorm workflows at various helm hook stages: post-install, pre-upgrade, post-upgrade. (#265) (by @cognifloyd)
 
 ## v0.80.0
 * Switch st2 to `v3.6` as a new default stable version (#274)

Chart.yaml

@@ -2,12 +2,12 @@ apiVersion: v2
 # StackStorm version which refers to Docker images tag
 appVersion: "3.6"
 name: stackstorm-ha
-version: 0.80.0
+version: 0.90.0
 description: StackStorm K8s Helm Chart, optimized for running StackStorm in HA environment.
 home: https://stackstorm.com/
 icon: https://landscape.cncf.io/logos/stack-storm.svg
 source:
-  - https://github.com/stackstorm/stackstorm-ha
+  - https://github.com/stackstorm/stackstorm-k8s
 keywords:
   - st2
   - stackstorm

README.md

@@ -123,6 +123,11 @@ which in our case is `Redis`.
 st2sensorcontainer manages StackStorm sensors: It starts, stops and restarts them as subprocesses.
 By default, deployment is configured with `1` replica containing all the sensors.
 
+You can increase the number of `st2sensorcontainer` pods by increasing the number of deployments.
+The replicas count is still only `1` per deployment, but the sensors are distributed between the deployments using
+[Sensor Hash Range Partitioning](https://docs.stackstorm.com/reference/sensor_partitioning.html#hash).
+The hash ranges are calculated automatically based on the number of deployments.
+
 st2sensorcontainer also supports a more Docker-friendly single-sensor-per-container mode as a way of
 [Sensor Partitioning](https://docs.stackstorm.com/latest/reference/sensor_partitioning.html). This
 distributes the computing load between many pods and relies on K8s failover/reschedule mechanisms,
@@ -320,10 +325,40 @@ Grab all logs only for stackstorm backend services, excluding st2web and DB/MQ/r
 kubectl logs -l release=<release-name>,tier=backend
 ```
 
+## Running jobs before/after install, upgrade, or rollback
+WARNING: The feature described in this section is an Advanced feature that new users should not need.
+
+It may be convenient to run one or more `Job`(s) in your `stackstorm-ha` cluster to manage your release's life cycle.
+As the [Helm docs]() explain:
+
+> Helm provides a _hook_ mechanism to allow chart developers to intervene at certain points in a release's life cycle.
+
+The `jobs.extra_hooks` feature in this chart simplifies creating `Jobs` that Helm will run in its hooks.
+These jobs will use the same settings as any other job defined by this chart (eg image, annotations, pod placement).
+The `st2.conf` files and packs volumes will be mounted in the Job and the `st2` cli will be configured.
+This feature is primarily useful when you need to run a StackStorm workflow (with `st2 run ...`) after install,
+before/after upgrades, or before/after rollbacks.
+
+NOTE: The `jobs.extra_hooks` feature is very opinionated. If you need to to apply helm hooks to anything other than
+`Jobs`, or if these jobs do not meet your needs, then you will need to do so from a parent chart. For example, parent charts
+are much better suited to jobs that don't need access to the packs, configs, configmaps, and secrets that this chart provides.
+See "Extending this chart" below.
+
+These extra hooks jobs can be used for st2 installation-specific jobs like:
+
+- running a pre-upgrade st2 workflow that notifies on various channels that the upgrade is happening,
+- running post-upgrade smoke tests to ensure st2 can connect to vital services (vault, kubernetes, aws, etc),
+- running a pre-upgrade st2 workflow that pauses long-running workflows,
+- running a post-upgrade st2 workflow that resumes long-running workflows,
+- running one-time post-install configuration (such as generating dynamic secrets in the st2kv datastore),
+
+To configure the `jobs.extra_hooks`, set `jobs.extra_hooks` in your values file.
+Please refer to stackstorm-ha's default values.yaml file for examples.
+
 ## Extending this chart
 If you have any suggestions or ideas about how to extend this chart functionality,
-we welcome you to collaborate in [Issues](https://github.com/stackstorm/stackstorm-ha/issues)
-and contribute via [Pull Requests](https://github.com/stackstorm/stackstorm-ha/pulls).
+we welcome you to collaborate in [Issues](https://github.com/stackstorm/stackstorm-k8s/issues)
+and contribute via [Pull Requests](https://github.com/stackstorm/stackstorm-k8s/pulls).
 However if you need something very custom and specific to your infra that doesn't fit official chart plans,
 we strongly recommend you to create a parent Helm chart with custom K8s objects and referencing `stackstorm-ha` chart
 as a child dependency.