From bfb040ec05961ab0289d536aaac6926c8031934c Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Tue, 17 Sep 2024 16:29:41 +0200 Subject: [PATCH 1/9] Add descriptions --- .../modules/commons-operator/pages/index.adoc | 19 +++++++++++-- .../commons-operator/pages/installation.adoc | 10 +++---- .../pages/pod-enrichment.adoc | 18 ++++++------- .../pages/reference/crds.adoc | 2 +- .../commons-operator/pages/restarter.adoc | 27 +++++++++---------- .../modules/commons-operator/pages/usage.adoc | 4 +-- 6 files changed, 45 insertions(+), 35 deletions(-) diff --git a/docs/modules/commons-operator/pages/index.adoc b/docs/modules/commons-operator/pages/index.adoc index 9852f8f..d584ded 100644 --- a/docs/modules/commons-operator/pages/index.adoc +++ b/docs/modules/commons-operator/pages/index.adoc @@ -1,4 +1,5 @@ = Stackable Commons Operator +:description: Learn about shared objects (AuthenticationClass, S3Bucket and S3Connection) and shared functionality (Pod enrichting, restarting) on the Stackable Data Platform. :github: https://github.com/stackabletech/commons-operator/ :crd: {crd-docs-base-url}/commons-operator/{crd-docs-version}/ @@ -6,5 +7,19 @@ * {github}[GitHub {external-link-icon}^] * {crd}[CRD documentation {external-link-icon}^] -This is an operator for Kubernetes that provides and manages common objects of the Stackable Data Platform. -Have a look at the CRD documentation linked above to see the the custom resources that this operator deploys. +Unlike most other operators on the Stackable Data Platform, the Stackable commons operator does not manage any particular Stacklet, but rather provides shared functionality and resource definitions that are used by the other operators. + +== Common resources + +The _commons_ Kubernetes operator for the Stackable Data Platform provides and manages resources that are not tied to any particular data product, but usually shared among multiple products. +These are the _AuthenticationClass_ for configuring authentication mechanisms, and the _S3Bucket_ and _S3Connection_ for configuring S3 object storage. +Usually all tools in a data platform that require authentication will be configured to use the same authentication mechanism, so the AuthenticationClass provides a single configuration point for authentication methods, which can then be referenced in multiple data products. +Likewise, the S3 related objects provide single configuration points for S3 storage, which is then referenced in multiple products. + +Because these objects are used throughout the whole platform, the documentation is also mostly found at the platform level. +Read the xref:concepts:authentication.adoc[authentication] concepts page to learn about the AuthenticationClass and learn about S3 on the xref:concepts:s3.adoc[S3 resources] page. +You can also find the full CRD refererence documentation for all objects linked above. + +== Shared functionality: Pod enriching and restarting + +The commons operator also xref:pod-enrichment.adoc[adds additional information to Pods] which would otherwise have to be inferred and takes care of xref:restarter.adoc[restarting] Pods when necessary, for example to reload configuration settings in Pods that don't support hot reloading. diff --git a/docs/modules/commons-operator/pages/installation.adoc b/docs/modules/commons-operator/pages/installation.adoc index 33ac28a..0d96d19 100644 --- a/docs/modules/commons-operator/pages/installation.adoc +++ b/docs/modules/commons-operator/pages/installation.adoc @@ -2,9 +2,8 @@ There are two ways to run the Stackable Commons Operator: -1. Helm managed Docker container deployment on Kubernetes - -2. Build from source +. Helm managed Docker container deployment on Kubernetes +. Build from source == Prerequisites @@ -20,7 +19,8 @@ Resource sizing depends on cluster type(s), usage and scope, but as a starting p * 256MB RAM == Helm -Helm allows you to download and deploy Stackable operators on Kubernetes and is by far the easiest installation method. First ensure that you have installed the Stackable Operators Helm repository: +Helm allows you to download and deploy Stackable operators on Kubernetes and is by far the easiest installation method. +First ensure that you have installed the Stackable Operators Helm repository: [source,console] ---- @@ -38,7 +38,7 @@ Helm will deploy the operator in Kubernetes and apply the CRDs. == Building the operator from source -After cloning the commons-operator repo you can use the following command to apply the CRDs: +After cloning the commons-operator repository you can use the following command to apply the CRDs: [source,console] ---- diff --git a/docs/modules/commons-operator/pages/pod-enrichment.adoc b/docs/modules/commons-operator/pages/pod-enrichment.adoc index 07853a3..e695e93 100644 --- a/docs/modules/commons-operator/pages/pod-enrichment.adoc +++ b/docs/modules/commons-operator/pages/pod-enrichment.adoc @@ -1,19 +1,17 @@ -= Pod Enricher += Pod enricher +:description: Automatically enrich Pods with Node address info using Stackable Commons Operator by enabling the enrichment label. -The Stackable Commons Operator automatically adds commonly used information to `Pod` objects, which -would otherwise have to be inferred by traversing the Kubernetes object graph. +The Stackable Commons Operator automatically adds commonly used information to Pod objects, which would otherwise have to be inferred by traversing the Kubernetes object graph. == Usage -The pod enricher is only enabled for `Pod` objects that set the label `enrichment.stackable.tech/enabled` to `true`. +To enable the pod enricher for a Pod, set the label `enrichment.stackable.tech/enabled` to `true`. -== Node Address +== Which information is added to the Pod? Annotation:: `enrichment.stackable.tech/node-address` -The hostname or IP address of the `Node` that the `Pod` is scheduled to run on. -Compared to `Pod.status.nodeIP`, this can also (but doesn't have to) be a hostname, and prefers -externally routable addresses. +The hostname or IP address of the Node that the Pod is scheduled to run on. +Compared to `Pod.status.nodeIP`, this can also (but doesn't have to) be a hostname, and prefers externally routable addresses. -This is intended to be used for components that need to register an accessible address (such as Kafka brokers, -or HDFS DataNodes). +This is intended to be used for components that need to register an accessible address (such as Kafka brokers, or HDFS DataNodes). diff --git a/docs/modules/commons-operator/pages/reference/crds.adoc b/docs/modules/commons-operator/pages/reference/crds.adoc index 786178e..ed1eaec 100644 --- a/docs/modules/commons-operator/pages/reference/crds.adoc +++ b/docs/modules/commons-operator/pages/reference/crds.adoc @@ -1,3 +1,3 @@ -= CRD Reference += CRD reference Find all CRD reference for the Stackable Commons Operator at: {crd-docs-base-url}/commons-operator/{crd-docs-version}. diff --git a/docs/modules/commons-operator/pages/restarter.adoc b/docs/modules/commons-operator/pages/restarter.adoc index 6094278..827a7ef 100644 --- a/docs/modules/commons-operator/pages/restarter.adoc +++ b/docs/modules/commons-operator/pages/restarter.adoc @@ -1,29 +1,26 @@ = Restarter +:description: Automatically restart Pods or StatefulSets using Stackable Commons Operator based on expiration dates or stale configurations. -The Stackable Commons Operator can automatically restart `Pod` objects based on certain criteria. This can be applied to -either the `Pod` or certain controller objects (such as `StatefulSet`). +The Stackable Commons Operator can automatically restart Pod objects based on certain criteria. +This can be applied to either the Pod or certain controller objects (such as StatefulSet). -== `Pod` +== Pod -Pods are evicted when any of their restart criteria (listed below) expire, with the expectation that their owning controller -is then responsible for restarting them. +Pods are evicted when any of their restart criteria (listed below) expire, with the expectation that their owning controller is then responsible for restarting them. -Because they are evicted rather than deleted, this process should respect `PodDisruptionBudget` constraints, allowing users -to ensure that clusters are restarted gracefully. +Because they are evicted rather than deleted, this process should respect PodDisruptionBudget constraints, allowing users to ensure that clusters are restarted gracefully. === Expiration date Annotation:: `restarter.stackable.tech/expires-at.\{tag\}` -Pods can be configured to expire at a certain point in time. In this case, the `Pod` should have the annotation -`restarter.stackable.tech/expires-at.\{tag\}` set to a datetime formatted according to RFC 3339 (such as -`"2022-04-21T13:24:15.225774724+00:00"`). +Pods can be configured to expire at a certain point in time. +In this case, the Pod should have the annotation `restarter.stackable.tech/expires-at.\{tag\}` set to a datetime formatted according to RFC 3339 (such as `"2022-04-21T13:24:15.225774724+00:00"`). `\{tag\}` should be a deterministic but unique ID identifying the reason for the expiry. -Multiple `expires-at` annotations can be set on the same `Pod`, in which case the *earliest* expiration datetime -takes precedence. +Multiple `expires-at` annotations can be set on the same Pod, in which case the *earliest* expiration datetime takes precedence. -== `StatefulSet` +== StatefulSet StatefulSets are rolling-restarted when any of their restart criteria (listed below) expire. @@ -31,5 +28,5 @@ StatefulSets are rolling-restarted when any of their restart criteria (listed be Label:: `restarter.stackable.tech/enabled` -StatefulSets can be configured to restart when any referenced configuration object (`ConfigMap` or `Secret`) changes. -To enable this, set the `restarter.stackable.tech/enabled` label on the `StatefulSet` to `true`. +StatefulSets can be configured to restart when any referenced configuration object (ConfigMap or Secret) changes. +To enable this, set the `restarter.stackable.tech/enabled` label on the StatefulSet to `true`. diff --git a/docs/modules/commons-operator/pages/usage.adoc b/docs/modules/commons-operator/pages/usage.adoc index 9d6ffd4..e8884d6 100644 --- a/docs/modules/commons-operator/pages/usage.adoc +++ b/docs/modules/commons-operator/pages/usage.adoc @@ -7,8 +7,8 @@ The commons-operator is used for multiple purposes: |Concept|Purpose |xref:restarter.adoc[] -|A controller that watches `Pod` objects and their controllers and restarts them when required +|A controller that watches Pod objects and their controllers and restarts them when required. |xref:pod-enrichment.adoc[] -|A controller that adds commonly used information to `Pod` objects +|A controller that adds commonly used information to Pod objects. |=== From f2d90751230b2efd6230e8480ec284f7dba5b34d Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Tue, 17 Sep 2024 17:00:45 +0200 Subject: [PATCH 2/9] Update installation instructions --- .../commons-operator/pages/installation.adoc | 63 +++++++++++-------- 1 file changed, 37 insertions(+), 26 deletions(-) diff --git a/docs/modules/commons-operator/pages/installation.adoc b/docs/modules/commons-operator/pages/installation.adoc index 0d96d19..e9e5360 100644 --- a/docs/modules/commons-operator/pages/installation.adoc +++ b/docs/modules/commons-operator/pages/installation.adoc @@ -1,9 +1,5 @@ = Installation - -There are two ways to run the Stackable Commons Operator: - -. Helm managed Docker container deployment on Kubernetes -. Build from source +:description: Install the Stackable Commons Operator with either stackablectl or Helm. == Prerequisites @@ -18,36 +14,51 @@ Resource sizing depends on cluster type(s), usage and scope, but as a starting p * 0.2 cores (e.g. i5 or similar) * 256MB RAM -== Helm -Helm allows you to download and deploy Stackable operators on Kubernetes and is by far the easiest installation method. -First ensure that you have installed the Stackable Operators Helm repository: +== Installation -[source,console] ----- -$ helm repo add stackable https://repo.stackable.tech/repository/helm-stable/ ----- +There are two ways to deploy the Commons operator into your Kubernetes cluster. -Then install the Stackable Commons Operator +. Using xref:management:stackablectl:index.adoc[]. +. Using Helm. + +[tabs] +==== +stackablectl:: ++ +-- +`stackablectl` is the command line tool to interact with Stackable operators and our recommended way to install Operators. +Follow the xref:management:stackablectl:installation.adoc[installation steps] for your platform. + +After you have installed `stackablectl`, use it to install the Commons Operator: [source,console] ----- -$ helm install commons-operator stackable/commons-operator ----- +$ stackablectl operator install commons=24.7.0 -Helm will deploy the operator in Kubernetes and apply the CRDs. +The tool will show: + +[source] +Installed commons=24.7.0 operator -== Building the operator from source +TIP: Consult the xref:management:stackablectl:quickstart.adoc[] to learn more about how to use `stackablectl`. For +example, you can use the `--cluster kind` flag to create a Kubernetes cluster with link:https://kind.sigs.k8s.io/[kind]. +-- -After cloning the commons-operator repository you can use the following command to apply the CRDs: +Helm:: ++ +-- +Helm allows you to download and deploy Stackable operators on Kubernetes. +First ensure that you have installed the Stackable Operators Helm repository: [source,console] ----- -$ cargo run -- crd | kubectl apply -f - ----- +$ helm repo add stackable \ + https://repo.stackable.tech/repository/helm-stable/ -Use the following command to run the operator: +Then install the Stackable Commons Operator [source,console] ----- -$ cargo run -- run ----- +$ helm install --wait commons-operator \ + stackable-stable/commons-operator --version 24.7.0 + +Helm will deploy the operator in Kubernetes and apply the CRDs. +-- +==== \ No newline at end of file From 6a69fdf6a6bc2b51f800ed2d83540e778901f499 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Tue, 17 Sep 2024 17:03:39 +0200 Subject: [PATCH 3/9] Add newline to end of file --- docs/modules/commons-operator/pages/installation.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/commons-operator/pages/installation.adoc b/docs/modules/commons-operator/pages/installation.adoc index e9e5360..f4f4832 100644 --- a/docs/modules/commons-operator/pages/installation.adoc +++ b/docs/modules/commons-operator/pages/installation.adoc @@ -61,4 +61,4 @@ $ helm install --wait commons-operator \ Helm will deploy the operator in Kubernetes and apply the CRDs. -- -==== \ No newline at end of file +==== From 6797b06990c7737cc9d7cff24704e8924c9d6c4b Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Wed, 18 Sep 2024 15:12:37 +0200 Subject: [PATCH 4/9] Update docs/modules/commons-operator/pages/index.adoc Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com> --- docs/modules/commons-operator/pages/index.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/commons-operator/pages/index.adoc b/docs/modules/commons-operator/pages/index.adoc index d584ded..2cb508c 100644 --- a/docs/modules/commons-operator/pages/index.adoc +++ b/docs/modules/commons-operator/pages/index.adoc @@ -9,7 +9,7 @@ Unlike most other operators on the Stackable Data Platform, the Stackable commons operator does not manage any particular Stacklet, but rather provides shared functionality and resource definitions that are used by the other operators. -== Common resources +== Shared resources The _commons_ Kubernetes operator for the Stackable Data Platform provides and manages resources that are not tied to any particular data product, but usually shared among multiple products. These are the _AuthenticationClass_ for configuring authentication mechanisms, and the _S3Bucket_ and _S3Connection_ for configuring S3 object storage. From 6c06084e0626bec5e92708b9a8ddfcb11c75c4c5 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Wed, 18 Sep 2024 15:12:48 +0200 Subject: [PATCH 5/9] Update docs/modules/commons-operator/pages/index.adoc Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com> --- docs/modules/commons-operator/pages/index.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/commons-operator/pages/index.adoc b/docs/modules/commons-operator/pages/index.adoc index 2cb508c..5b29000 100644 --- a/docs/modules/commons-operator/pages/index.adoc +++ b/docs/modules/commons-operator/pages/index.adoc @@ -20,6 +20,6 @@ Because these objects are used throughout the whole platform, the documentation Read the xref:concepts:authentication.adoc[authentication] concepts page to learn about the AuthenticationClass and learn about S3 on the xref:concepts:s3.adoc[S3 resources] page. You can also find the full CRD refererence documentation for all objects linked above. -== Shared functionality: Pod enriching and restarting +== Shared functionality The commons operator also xref:pod-enrichment.adoc[adds additional information to Pods] which would otherwise have to be inferred and takes care of xref:restarter.adoc[restarting] Pods when necessary, for example to reload configuration settings in Pods that don't support hot reloading. From a2fdf99f18718b38c7a77c006b46aa168f4bf244 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Wed, 18 Sep 2024 15:13:16 +0200 Subject: [PATCH 6/9] Update docs/modules/commons-operator/pages/index.adoc Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com> --- docs/modules/commons-operator/pages/index.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/modules/commons-operator/pages/index.adoc b/docs/modules/commons-operator/pages/index.adoc index 5b29000..9e14f63 100644 --- a/docs/modules/commons-operator/pages/index.adoc +++ b/docs/modules/commons-operator/pages/index.adoc @@ -22,4 +22,5 @@ You can also find the full CRD refererence documentation for all objects linked == Shared functionality -The commons operator also xref:pod-enrichment.adoc[adds additional information to Pods] which would otherwise have to be inferred and takes care of xref:restarter.adoc[restarting] Pods when necessary, for example to reload configuration settings in Pods that don't support hot reloading. +The commons operator implements functionality that is shared by all other Stackable operators. +For example, it xref:pod-enrichment.adoc[adds additional information to Pods] which would otherwise have to be inferred and takes care of xref:restarter.adoc[restarting] Pods when necessary, for example to reload configuration settings in products that don't support hot reloading. From 64ea5453f161c66ce7b531d142b6fb869db946a3 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Wed, 18 Sep 2024 15:13:29 +0200 Subject: [PATCH 7/9] Update docs/modules/commons-operator/pages/pod-enrichment.adoc Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com> --- docs/modules/commons-operator/pages/pod-enrichment.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/commons-operator/pages/pod-enrichment.adoc b/docs/modules/commons-operator/pages/pod-enrichment.adoc index e695e93..05eaa08 100644 --- a/docs/modules/commons-operator/pages/pod-enrichment.adoc +++ b/docs/modules/commons-operator/pages/pod-enrichment.adoc @@ -1,5 +1,5 @@ = Pod enricher -:description: Automatically enrich Pods with Node address info using Stackable Commons Operator by enabling the enrichment label. +:description: Use Labels to instruct the Stackable Commons Operator to enrich Pods with Kubernetes node addresses The Stackable Commons Operator automatically adds commonly used information to Pod objects, which would otherwise have to be inferred by traversing the Kubernetes object graph. From 759da063e3ddf16cfc504d08283d7c98528c99cf Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Wed, 18 Sep 2024 15:13:41 +0200 Subject: [PATCH 8/9] Update docs/modules/commons-operator/pages/restarter.adoc Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com> --- docs/modules/commons-operator/pages/restarter.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/commons-operator/pages/restarter.adoc b/docs/modules/commons-operator/pages/restarter.adoc index 827a7ef..e2e2c8b 100644 --- a/docs/modules/commons-operator/pages/restarter.adoc +++ b/docs/modules/commons-operator/pages/restarter.adoc @@ -28,5 +28,5 @@ StatefulSets are rolling-restarted when any of their restart criteria (listed be Label:: `restarter.stackable.tech/enabled` -StatefulSets can be configured to restart when any referenced configuration object (ConfigMap or Secret) changes. +The operator can restart StatefulSets when any referenced configuration object (ConfigMap or Secret) changes. To enable this, set the `restarter.stackable.tech/enabled` label on the StatefulSet to `true`. From 2a4bf1a2327758889177989e479a06cca1b5bef4 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Wed, 18 Sep 2024 15:13:54 +0200 Subject: [PATCH 9/9] Update docs/modules/commons-operator/pages/installation.adoc Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com> --- docs/modules/commons-operator/pages/installation.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/commons-operator/pages/installation.adoc b/docs/modules/commons-operator/pages/installation.adoc index f4f4832..b5174f9 100644 --- a/docs/modules/commons-operator/pages/installation.adoc +++ b/docs/modules/commons-operator/pages/installation.adoc @@ -26,7 +26,7 @@ There are two ways to deploy the Commons operator into your Kubernetes cluster. stackablectl:: + -- -`stackablectl` is the command line tool to interact with Stackable operators and our recommended way to install Operators. +`stackablectl` is the recommended command line tool to install and manage with Stackable operators. Follow the xref:management:stackablectl:installation.adoc[installation steps] for your platform. After you have installed `stackablectl`, use it to install the Commons Operator: