Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions docs/modules/commons-operator/pages/index.adoc
Original file line number Diff line number Diff line change
@@ -1,10 +1,26 @@
= Stackable Commons Operator
:description: Learn about shared objects (AuthenticationClass, S3Bucket and S3Connection) and shared functionality (Pod enrichting, restarting) on the Stackable Data Platform.
:github: https://github.com/stackabletech/commons-operator/
:crd: {crd-docs-base-url}/commons-operator/{crd-docs-version}/

[.link-bar]
* {github}[GitHub {external-link-icon}^]
* {crd}[CRD documentation {external-link-icon}^]

This is an operator for Kubernetes that provides and manages common objects of the Stackable Data Platform.
Have a look at the CRD documentation linked above to see the the custom resources that this operator deploys.
Unlike most other operators on the Stackable Data Platform, the Stackable commons operator does not manage any particular Stacklet, but rather provides shared functionality and resource definitions that are used by the other operators.

== Shared resources

The _commons_ Kubernetes operator for the Stackable Data Platform provides and manages resources that are not tied to any particular data product, but usually shared among multiple products.
These are the _AuthenticationClass_ for configuring authentication mechanisms, and the _S3Bucket_ and _S3Connection_ for configuring S3 object storage.
Usually all tools in a data platform that require authentication will be configured to use the same authentication mechanism, so the AuthenticationClass provides a single configuration point for authentication methods, which can then be referenced in multiple data products.
Likewise, the S3 related objects provide single configuration points for S3 storage, which is then referenced in multiple products.

Because these objects are used throughout the whole platform, the documentation is also mostly found at the platform level.
Read the xref:concepts:authentication.adoc[authentication] concepts page to learn about the AuthenticationClass and learn about S3 on the xref:concepts:s3.adoc[S3 resources] page.
You can also find the full CRD refererence documentation for all objects linked above.

== Shared functionality

The commons operator implements functionality that is shared by all other Stackable operators.
For example, it xref:pod-enrichment.adoc[adds additional information to Pods] which would otherwise have to be inferred and takes care of xref:restarter.adoc[restarting] Pods when necessary, for example to reload configuration settings in products that don't support hot reloading.
63 changes: 37 additions & 26 deletions docs/modules/commons-operator/pages/installation.adoc
Original file line number Diff line number Diff line change
@@ -1,10 +1,5 @@
= Installation

There are two ways to run the Stackable Commons Operator:

1. Helm managed Docker container deployment on Kubernetes

2. Build from source
:description: Install the Stackable Commons Operator with either stackablectl or Helm.

== Prerequisites

Expand All @@ -19,35 +14,51 @@ Resource sizing depends on cluster type(s), usage and scope, but as a starting p
* 0.2 cores (e.g. i5 or similar)
* 256MB RAM

== Helm
Helm allows you to download and deploy Stackable operators on Kubernetes and is by far the easiest installation method. First ensure that you have installed the Stackable Operators Helm repository:
== Installation

[source,console]
----
$ helm repo add stackable https://repo.stackable.tech/repository/helm-stable/
----
There are two ways to deploy the Commons operator into your Kubernetes cluster.

Then install the Stackable Commons Operator
. Using xref:management:stackablectl:index.adoc[].
. Using Helm.

[tabs]
====
stackablectl::
+
--
`stackablectl` is the recommended command line tool to install and manage with Stackable operators.
Follow the xref:management:stackablectl:installation.adoc[installation steps] for your platform.

After you have installed `stackablectl`, use it to install the Commons Operator:

[source,console]
----
$ helm install commons-operator stackable/commons-operator
----
$ stackablectl operator install commons=24.7.0

Helm will deploy the operator in Kubernetes and apply the CRDs.
The tool will show:

[source]
Installed commons=24.7.0 operator

== Building the operator from source
TIP: Consult the xref:management:stackablectl:quickstart.adoc[] to learn more about how to use `stackablectl`. For
example, you can use the `--cluster kind` flag to create a Kubernetes cluster with link:https://kind.sigs.k8s.io/[kind].
--

After cloning the commons-operator repo you can use the following command to apply the CRDs:
Helm::
+
--
Helm allows you to download and deploy Stackable operators on Kubernetes.
First ensure that you have installed the Stackable Operators Helm repository:

[source,console]
----
$ cargo run -- crd | kubectl apply -f -
----
$ helm repo add stackable \
https://repo.stackable.tech/repository/helm-stable/

Use the following command to run the operator:
Then install the Stackable Commons Operator

[source,console]
----
$ cargo run -- run
----
$ helm install --wait commons-operator \
stackable-stable/commons-operator --version 24.7.0

Helm will deploy the operator in Kubernetes and apply the CRDs.
--
====
18 changes: 8 additions & 10 deletions docs/modules/commons-operator/pages/pod-enrichment.adoc
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
= Pod Enricher
= Pod enricher
:description: Use Labels to instruct the Stackable Commons Operator to enrich Pods with Kubernetes node addresses

The Stackable Commons Operator automatically adds commonly used information to `Pod` objects, which
would otherwise have to be inferred by traversing the Kubernetes object graph.
The Stackable Commons Operator automatically adds commonly used information to Pod objects, which would otherwise have to be inferred by traversing the Kubernetes object graph.

== Usage

The pod enricher is only enabled for `Pod` objects that set the label `enrichment.stackable.tech/enabled` to `true`.
To enable the pod enricher for a Pod, set the label `enrichment.stackable.tech/enabled` to `true`.

== Node Address
== Which information is added to the Pod?

Annotation:: `enrichment.stackable.tech/node-address`

The hostname or IP address of the `Node` that the `Pod` is scheduled to run on.
Compared to `Pod.status.nodeIP`, this can also (but doesn't have to) be a hostname, and prefers
externally routable addresses.
The hostname or IP address of the Node that the Pod is scheduled to run on.
Compared to `Pod.status.nodeIP`, this can also (but doesn't have to) be a hostname, and prefers externally routable addresses.

This is intended to be used for components that need to register an accessible address (such as Kafka brokers,
or HDFS DataNodes).
This is intended to be used for components that need to register an accessible address (such as Kafka brokers, or HDFS DataNodes).
2 changes: 1 addition & 1 deletion docs/modules/commons-operator/pages/reference/crds.adoc
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
= CRD Reference
= CRD reference

Find all CRD reference for the Stackable Commons Operator at: {crd-docs-base-url}/commons-operator/{crd-docs-version}.
27 changes: 12 additions & 15 deletions docs/modules/commons-operator/pages/restarter.adoc
Original file line number Diff line number Diff line change
@@ -1,35 +1,32 @@
= Restarter
:description: Automatically restart Pods or StatefulSets using Stackable Commons Operator based on expiration dates or stale configurations.

The Stackable Commons Operator can automatically restart `Pod` objects based on certain criteria. This can be applied to
either the `Pod` or certain controller objects (such as `StatefulSet`).
The Stackable Commons Operator can automatically restart Pod objects based on certain criteria.
This can be applied to either the Pod or certain controller objects (such as StatefulSet).

== `Pod`
== Pod

Pods are evicted when any of their restart criteria (listed below) expire, with the expectation that their owning controller
is then responsible for restarting them.
Pods are evicted when any of their restart criteria (listed below) expire, with the expectation that their owning controller is then responsible for restarting them.

Because they are evicted rather than deleted, this process should respect `PodDisruptionBudget` constraints, allowing users
to ensure that clusters are restarted gracefully.
Because they are evicted rather than deleted, this process should respect PodDisruptionBudget constraints, allowing users to ensure that clusters are restarted gracefully.

=== Expiration date

Annotation:: `restarter.stackable.tech/expires-at.\{tag\}`

Pods can be configured to expire at a certain point in time. In this case, the `Pod` should have the annotation
`restarter.stackable.tech/expires-at.\{tag\}` set to a datetime formatted according to RFC 3339 (such as
`"2022-04-21T13:24:15.225774724+00:00"`).
Pods can be configured to expire at a certain point in time.
In this case, the Pod should have the annotation `restarter.stackable.tech/expires-at.\{tag\}` set to a datetime formatted according to RFC 3339 (such as `"2022-04-21T13:24:15.225774724+00:00"`).
`\{tag\}` should be a deterministic but unique ID identifying the reason for the expiry.

Multiple `expires-at` annotations can be set on the same `Pod`, in which case the *earliest* expiration datetime
takes precedence.
Multiple `expires-at` annotations can be set on the same Pod, in which case the *earliest* expiration datetime takes precedence.

== `StatefulSet`
== StatefulSet

StatefulSets are rolling-restarted when any of their restart criteria (listed below) expire.

=== Stale configuration

Label:: `restarter.stackable.tech/enabled`

StatefulSets can be configured to restart when any referenced configuration object (`ConfigMap` or `Secret`) changes.
To enable this, set the `restarter.stackable.tech/enabled` label on the `StatefulSet` to `true`.
The operator can restart StatefulSets when any referenced configuration object (ConfigMap or Secret) changes.
To enable this, set the `restarter.stackable.tech/enabled` label on the StatefulSet to `true`.
4 changes: 2 additions & 2 deletions docs/modules/commons-operator/pages/usage.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ The commons-operator is used for multiple purposes:
|Concept|Purpose

|xref:restarter.adoc[]
|A controller that watches `Pod` objects and their controllers and restarts them when required
|A controller that watches Pod objects and their controllers and restarts them when required.

|xref:pod-enrichment.adoc[]
|A controller that adds commonly used information to `Pod` objects
|A controller that adds commonly used information to Pod objects.
|===
Loading