[ECK] Update documentation for 3.1.0

deploy-manage/deploy/cloud-on-k8s.md

@@ -69,10 +69,23 @@ This section outlines the supported Kubernetes and {{stack}} versions for ECK. C
 
 ECK is compatible with the following Kubernetes distributions and related technologies:
 
+::::{tab-set}
+
+:::{tab-item} ECK 3.1
+* Kubernetes 1.29-1.33
+* OpenShift 4.15-4.19
+* Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), and Amazon Elastic Kubernetes Service (EKS)
+* Helm: {{eck_helm_minimum_version}}+
+:::
+
+:::{tab-item} ECK 3.0
 * Kubernetes 1.28-1.32
 * OpenShift 4.14-4.18
 * Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), and Amazon Elastic Kubernetes Service (EKS)
 * Helm: {{eck_helm_minimum_version}}+
+:::
+
+::::
 
 ECK should work with all conformant **installers** listed in these [FAQs](https://github.com/cncf/k8s-conformance/blob/master/faq.md#what-is-a-distribution-hosted-platform-and-an-installer). Distributions include source patches and so may not work as-is with ECK.
 

deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md

@@ -146,8 +146,23 @@ By default, every reference targets all instances in your {{es}}, {{kib}} and {{
 
 ## Customize {{agent}} configuration [k8s-elastic-agent-fleet-configuration-custom-configuration]
 
-In contrast to {{agents}} in standalone mode, the configuration is managed through {{fleet}}, and it cannot be defined through `config` or `configRef` elements.
+In contrast to {{agents}} in standalone mode, the configuration is managed through {{fleet}}, and it cannot be defined through `config` or `configRef` elements with a few exceptions.
 
+One of those exceptions is the configuration of providers as described in [advanced Agent configuration managed by Fleet](/reference/fleet/advanced-kubernetes-managed-by-fleet.md). When {{agent}} is managed by {{fleet}} and is orchestrated by ECK, the configuration of providers can simply be done through the `.spec.config` element in the Agent resource as of {applies_to}`stack: ga 8.13`:
+
+```yaml
+apiVersion: agent.k8s.elastic.co/v1alpha1
+kind: Agent
+metadata:
+  name: elastic-agent
+spec:
+  config:
+    fleet:
+      enabled: true
+    providers.kubernetes:
+      add_resource_metadata:
+        deployment: true
+```
 
 ## Upgrade the {{agent}} specification [k8s-elastic-agent-fleet-configuration-upgrade-specification]
 

deploy-manage/deploy/cloud-on-k8s/propagate-labels-annotations.md

@@ -0,0 +1,86 @@
+---
+applies_to:
+  deployment:
+    eck: preview 3.1
+products:
+  - id: cloud-kubernetes
+---
+
+# Propagate Labels and Annotations [k8s-propagate-labels-annotations]
+
+Starting with version `3.1.0`, {{eck}} supports propagating labels and annotations from the parent resource to the child resources it creates. This can be used on all custom resources managed by ECK, such as {{eck_resources_list}}.
+
+The example below demonstrates how to use this feature on a {{es}} cluster, however, as mentioned above, this can be also applied to any custom resource managed by {{eck}}.
+
+```yaml
+apiVersion: elasticsearch.k8s.elastic.co/v1
+kind: Elasticsearch
+metadata:
+  annotations:
+    # Some custom annotations to be propagated to resources created by the operator.
+    my-annotation1: "my-annotation1-value"
+    my-annotation2: "my-annotation2-value"
+    # Instructions for the operator to propagate these annotations and labels to resources it creates.
+    eck.k8s.alpha.elastic.co/propagate-annotations: "my-annotation1, my-annotation2"
+    eck.k8s.alpha.elastic.co/propagate-labels: "my-label1, my-label2"
+  labels:
+    # Some custom labels to be propagated to resources created by the operator.
+    my-label1: "my-label1-value"
+    my-label2: "my-label2-value"
+  name: elasticsearch-sample
+spec:
+  version: 9.1.0
+  nodeSets:
+    - name: default
+      config:
+        # this allows ES to run on nodes even if their vm.max_map_count has not been increased, at a performance cost
+        node.store.allow_mmap: false
+      count: 1
+```
+
+The custom labels and annotations specified in the `metadata` section of the parent resource will be propagated to all child resources created by {{eck}}, such as StatefulSets, Pods, Services, and Secrets. This ensures that all resources have consistent metadata, which can be useful for filtering, monitoring, and managing resources in Kubernetes:
+
+```sh
+kubectl get sts,pods,svc -l my-label1=my-label1-value,my-label2=my-label2-value
+```
+
+```sh
+NAME                                               READY   AGE
+statefulset.apps/elasticsearch-sample-es-default   1/1     4m10s
+
+NAME                                    READY   STATUS    RESTARTS   AGE
+pod/elasticsearch-sample-es-default-0   1/1     Running   0          4m9s
+
+NAME                                            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
+service/elasticsearch-sample-es-default         ClusterIP   None             <none>        9200/TCP   4m12s
+service/elasticsearch-sample-es-http            ClusterIP   XX.XX.XX.XX      <none>        9200/TCP   4m14s
+service/elasticsearch-sample-es-internal-http   ClusterIP   XX.XX.XX.XX      <none>        9200/TCP   4m14s
+service/elasticsearch-sample-es-transport       ClusterIP   None             <none>        9300/TCP   4m14s
+```
+
+It is possible to use `*` as a wildcard to propagate all labels and annotations from the parent resource to the child resources. For example:
+
+```yaml
+apiVersion: elasticsearch.k8s.elastic.co/v1
+kind: Elasticsearch
+metadata:
+  annotations:
+    # Instructions for the operator to propagate all the annotations and labels to resources it creates.
+    eck.k8s.alpha.elastic.co/propagate-annotations: "*"
+    eck.k8s.alpha.elastic.co/propagate-labels: "*"
+  name: elasticsearch-sample
+spec:
+  version: 9.1.0
+  nodeSets:
+    - name: default
+      config:
+        # this allows ES to run on nodes even if their vm.max_map_count has not been increased, at a performance cost
+        node.store.allow_mmap: false
+      count: 1
+```
+
+::::{note}
+Note the following considerations when using this feature:
+* Propagated labels and annotations are not automatically deleted. If you want to remove them from the child resources, you need to do so manually or use a cleanup script.
+* To prevent conflicts, some labels and annotations reserved for internal use by ECK or Kubernetes are not propagated. This is the case for labels and annotations that match `*.k8s.*.elastic.co/` and also `kubectl.kubernetes.io/last-applied-configuration`.
+::::

deploy-manage/toc.yml

@@ -224,6 +224,7 @@ toc:
                       - file: deploy/cloud-on-k8s/k8s-kibana-advanced-configuration.md
                       - file: deploy/cloud-on-k8s/k8s-kibana-plugins.md
                   - file: deploy/cloud-on-k8s/customize-pods.md
+                  - file: deploy/cloud-on-k8s/propagate-labels-annotations.md
                   - file: deploy/cloud-on-k8s/manage-compute-resources.md
                   - file: deploy/cloud-on-k8s/recipes.md
                   - file: deploy/cloud-on-k8s/connect-to-external-elastic-resources.md

docset.yml

@@ -1,3 +1,3 @@
 project: 'Elastic documentation'
 max_toc_depth: 2
 
@@ -280,8 +280,8 @@
   kib-pull:   "https://github.com/elastic/kibana/pull/"
   stack-version: "9.0.3"
   ece_version: "4.0.1"
-  eck_version: "3.0.0"
-  eck_release_branch: "3.0"
+  eck_version: "3.1.0"
+  eck_release_branch: "3.1"
   eck_helm_minimum_version: "3.2.0"
   eck_resources_list: "Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash"
   eck_resources_list_short: "APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash"

reference/fleet/advanced-kubernetes-managed-by-fleet.md

@@ -106,4 +106,5 @@ volumes:
 
 1. By default the manifests for {{agent}} managed by {{fleet}} have `hostNetwork:true`. In order to support multiple installations of {{agent}}s in the same node you should set `hostNetwork:false`. See this relevant [example](https://github.com/elastic/elastic-agent/tree/main/docs/manifests/hostnetwork) as described in [{{agent}} Manifests in order to support Kube-State-Metrics Sharding](https://github.com/elastic/elastic-agent/blob/main/docs/elastic-agent-ksm-sharding.md).
 2. The volume `/usr/share/elastic-agent/state` must remain mounted in [elastic-agent-managed-kubernetes.yaml](https://github.com/elastic/elastic-agent/blob/main/deploy/kubernetes/elastic-agent-managed-kubernetes.yaml), otherwise custom config map provided above will be overwritten.
+3. If {{agent}} is deployed through ECK, you can define the provider configuration in the `spec.config` field of the Kubernetes custom resource. Refer to [{{fleet}}-managed {{agent}} on ECK](/deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md) for details.