Allow specifying Helm values for operator installations

[NickLarsenNZ]

When installing stacks/demos, the operator charts are installed with default Helm values.

It would be good to be able to

• override operator Helm values in stacks (eg: specify the ListenerClass presets for listener-operator)
• install/upgrade operators/release with custom Helm values (but be careful since the values are not consistent across all operators).

Considerations:

• How to make this look nice (because the command line could get rather long if having to specify each operator's overrides).
• Should we make it look under overrides/ (or similar) for ${listener-name}.yaml?
  • Should there be a flag to opt-in/out so that if the overrides/ (or similar) exists, it can be igored?
• Should we allow inlined values overrides in the stack definition, or just use the above?

Also, see issue about listener-operator (not only for stacks/demos): #356

---

Implementation

• Add a new CLI argument --operator-values (which takes a file path).
• Deserialise the values file BTreeMap<String, Value> (where String is the operator name).
• For the operator being installed, extract the applicable values and blast them over to helm as a string.
• listener class preset order of precedence: Auto Discovery < from --operator-values < from --listener-class-preset.
  • We might need to refactor the listener-class-preset (feat: Support configuring ListenerClass preset (with sensible defaults) #414) code so that we can determine whether --listener-class-preset was explicitly set (which should override whatever is in the operator-values.yaml), or if the preset came from autodetection (which should be overridden by the preset defined in the operator-values.yaml for listener-operator (if it is set)). Maybe need something like:

    enum ListenerClassPreset {
        Explicit(String),
        AutoDiscovered(String),
    }

[NickLarsenNZ]

Brainstorming session

Some ideas:

Inlined in the stack definition:

This could be a combination of the two:

Inlined values

  example:
    stackableRelease: dev
    stackableOperators:
      - commons
      - name: listener
        values:
          preset: ephemeral-nodes
          labels:
            foo: bar
      - secret

Inlined references to values files.

  example:
    stackableRelease: dev
    stackableOperators:
      - commons
      - name: listener
        valuesFile: listener-operator-values.yaml
      - secret

Inlined references to a single values file

  example:
    stackableRelease: dev
    stackableOperators:
      - commons
      - listener
      - secret
    stackableOperatorValuesFile: operator-values.yaml

and:

# could we have common values that get merged into the below, or use yaml anchors

listener-operator:
  ...

secret-operator:
  ...

nifi-operator:
  ...

Via the CLI

Per operator values

stackablectl release install xxx --listener-operator-values=listener-operator-values.yaml --secret-operator-values=secret-operator-values.yaml ...

# or

stackablectl release install xxx --operator-values=listener=listener-operator-values.yaml,secret=secret-operator-values.yaml ...

Single values file with sub-values inside

Like how sub-charts are configured, even though we don't use subcharts

stackablectl release install xxx --operator-values=operator-values.yaml
stackablectl operator install xxx --operator-values=operator-values.yaml
stackablectl stack install xxx --operator-values=operator-values.yaml
stackablectl demo install xxx --operator-values=operator-values.yaml

values:

# could we have common values that get merged into the below, or use yaml anchors

listener-operator:
  ...

secret-operator:
  ...

nifi-operator:
  ...

I think it would be good to have a single file which contains values for each operator. That way we can make use of common value or yaml anchors for reusable settings (since many values will be common across operators).

Example values file (much like how subcharts work):

# could we have common values that get merged into the below, or use yaml anchors

listener-operator:
  csiProvisioner:
    externalProvisioner:
      image:
        repository: oci.stackable.tech/sdp/sig-storage/csi-provisioner
        tag: v9000

commons-operator:
  telemetry:
    consoleLog:
      enabled: true
      level: warn

nifi-operator:
  podAnnotations:
    foo: bar

This can then be used by either the CLI:

stackablectl release install xxx --operator-values=operator-values.yaml
stackablectl operator install xxx --operator-values=operator-values.yaml
stackablectl stack install xxx --operator-values=operator-values.yaml
stackablectl demo install xxx --operator-values=operator-values.yaml

or the stack file:

stacks:
  example:
    stackableRelease: dev
    stackableOperators:
      - commons
      - listener
      - nifi
    stackableOperatorValuesFile: operator-values.yaml
    # todo: inline values

Note

It would still be nice to have a way to provide helm values directly in the stack file though.
Should the struct for stackableOperators be changed to support that, or a new key which has a map of operators to values?

option and key names subject to change

[NickLarsenNZ]

During the on-site, we (@xeniape, @Techassi, @Maleware, myself and others around the table) discussed this.

More team members mentioned having needed this feature before, which gives it more weight.

We agreed to submit this as a piece of work (into the replenishment process), but only do the CLI args part and not worry about the stacks.yaml part until that becomes (more) desired.

So, I will refine the description and put it into the replenishment process.