Futher LH configuration

Author: YuryHrytsuk

charts/longhorn/README.md

@@ -1,19 +1,24 @@
-# Exaplaining misc. questions regarding Longhorn (LH)
+# Longhorn (LH) Knowledge Base
+
+### Can LH be used for critical services (e.g. Database)
+
+As of now we shall avoid using LH for critical services. Instead we should refer to a more reliable and easy-to-maintain solution (e.g. Application-Level replication [Postgres Operators], S3, ...)
+
+LH is using networking for keeping replicas in sync and IO-heavy workloads may overload it easily leading to unpredictable consequances. Before we manage to extensively monitor LH and scale properly on demand, we shall not use it for critical services or IO-heave services
 
 ### How does LH decide which node's disk to use as storage
 
-There are 3 ways (could be combined together) https://longhorn.io/kb/tip-only-use-storage-on-a-set-of-nodes/
-* storag tag feature https://longhorn.io/docs/1.8.1/nodes-and-volumes/nodes/storage-tags/
-* node selectors that will restrict LH to certain nodes only (and disks on these nodes)
-* using https://longhorn.io/docs/archives/1.2.2/references/settings/#create-default-disk-on-labeled-nodes
+It depends on configuration. There are 3 possibilities:
+* https://longhorn.io/kb/tip-only-use-storage-on-a-set-of-nodes/#deploy-longhorn-components-only-on-a-specific-set-of-nodes
 
-### Will LH pick up disks from a newly-added node
+### Will LH pick up storage from a newly-added node
 
-Use 2 articles below to get detailed information
-* https://longhorn.io/kb/tip-only-use-storage-on-a-set-of-nodes/
-* https://longhorn.io/docs/1.8.1/nodes-and-volumes/nodes/default-disk-and-node-config/
+By default LH will use storage on all (newly created as well) Nodes where it runs. If `createDefaultDiskLabeledNodes` is configured, then it depends on label of the node
 
-### Running LH on subset of Nodes (via nodeSelectors) has a limitation
+https://longhorn.io/kb/tip-only-use-storage-on-a-set-of-nodes/#deploy-longhorn-components-only-on-a-specific-set-of-nodes
 
-The workloads that use PV (from LH) can only run on the nodes where LH runs. Check the Result description in the article below:
-* https://longhorn.io/kb/tip-only-use-storage-on-a-set-of-nodes/#deploy-longhorn-components-only-on-a-specific-set-of-nodes
+### Can workloads be run on nodes where there is no LH
+
+They can as long as LH is not bound to specific nodes via `nodeSelector` or `systemManagedComponentsNodeSelector` settings. If LH is configure to run on specific nodes, then workloads can only be run on these nodes only.
+
+Source: https://longhorn.io/kb/tip-only-use-storage-on-a-set-of-nodes/#deploy-longhorn-components-only-on-a-specific-set-of-nodes

charts/longhorn/values.yaml.gotmpl

@@ -2,25 +2,31 @@
 # https://github.com/longhorn/longhorn/tree/v1.8.1/chart#values
 
 global:
-  # Warning: updating (after installation) will cause downtime
+  # Warning: updating node selectors (after installation) will cause downtime
+  # https://longhorn.io/docs/archives/1.2.2/advanced-resources/deploy/node-selector/#setting-up-node-selector-after-longhorn-has-been-installed
+  #
+  # Warning: using node selectors will restrict our workloads to the same nodes
+  # https://longhorn.io/kb/tip-only-use-storage-on-a-set-of-nodes/#deploy-longhorn-components-only-on-a-specific-set-of-nodes
   nodeSelector: {}
   systemManagedComponentsNodeSelector: {}
 
 defaultSettings:
   replicaAutoBalance: best-effort
 
-  # use dedicated folder (disk)
+  # control on which nodes LH will use disks
+  # use `node.longhorn.io/create-default-disk` node label for control
+  createDefaultDiskLabeledNodes: true
+  # use dedicated folder (disk) for storage
   defaultDataPath: /longhorn
 
   # https://longhorn.io/docs/1.8.1/best-practices/#minimal-available-storage-and-over-provisioning
   storageMinimalAvailablePercentage: 10
 
-  # Prevent LH deletion. For deletion refer to https://longhorn.io/docs/1.8.1/deploy/uninstall/#prerequisite
+  # Prevent LH deletion. Set to true if you want to delete LH
   deletingConfirmationFlag: false
 
-  # we rely on pre-configure disk that uses defaultDataPath
-  # and node selectors to decide on which node to run LH and its storage
-  createDefaultDiskLabeledNodes: false
+  # let replicas to be scheduled on the same node
+  replicaSoftAntiAffinity: false
 
 persistence:
   # use only for non-critical ops workloads
@@ -33,12 +39,6 @@ persistence:
   defaultClassReplicaCount: 2
   defaultFsType: xfs
 
-  # Configure which disks are candidates for replica placement (i.e. storage of data)
-  diskSelector: {}
-
-  # Configure which nodes are candidates for replica placement (i.e. storage of data)
-  defaultNodeSelector: {}
-
 resources: # https://longhorn.io/docs/1.8.1/best-practices/#minimum-recommended-hardware
     requests:
       cpu: 0.5