Deploy and Manage: ECK

deploy-manage/deploy/cloud-on-k8s.md

@@ -1,23 +1,93 @@
 ---
+applies:
+  eck: all
 mapped_urls:
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-overview.html
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-topics.html
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-supported.html
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s_learn_more_about_eck.html
 ---
 
-# Elastic Cloud on Kubernetes
+# Elastic Cloud on Kubernetes [k8s-overview]
 
-% What needs to be done: Refine
+Built on the Kubernetes Operator pattern, {{eck}} (ECK) extends the basic Kubernetes orchestration capabilities to support the setup and management of Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash on Kubernetes.
 
-% GitHub issue: https://github.com/elastic/docs-projects/issues/357
+## ECK overview
 
-% Scope notes: Maybe we can even leave it as it is.
+With Elastic Cloud on Kubernetes, you can streamline critical operations, such as:
 
-% Use migrated content from existing pages that map to this page:
+1. Managing and monitoring multiple clusters
+2. Scaling cluster capacity and storage
+3. Performing safe configuration changes through rolling upgrades
+4. Securing clusters with TLS certificates
+5. Setting up hot-warm-cold architectures with availability zone awareness
 
-% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-overview.md
-% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md
-%      Notes: redirect only
-% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-supported.md
-% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md
+This section provides everything you need to install, configure, and manage Elastic Stack applications with ECK, including:
+
+- [](./cloud-on-k8s/deploy-an-orchestrator.md): ECK installation methods and configuration options. Deploy ECK on managed Kubernetes platforms like GKE, AKS, and EKS, on self-managed Kubernetes clusters, on OpenShift, and even in air-gapped environments.
+- [](./cloud-on-k8s/manage-deployments.md): Handle {{es}} clusters and {{kib}} instances through ECK.
+- [](./cloud-on-k8s/orchestrate-other-elastic-applications.md): Run APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash on Kubernetes.
+- [](./cloud-on-k8s/tools-apis.md): A collection of tools and APIs available in ECK based environments.
+
+Other sections of the documentation include the following important topics around ECK:
+
+- [Logging and Monitoring](../monitor.md): Configure stack monitoring and logs forwarding with the help of ECK.
+- [Remote Clusters](../remote-clusters.md): Configure remote clusters on ECK.
+- [](../tools.md): Add snapshot repositories to your {{es}} clusters for automatic snapshots.
+- [Security](../security.md): Secure communications, manage HTTP certificates, or add secure settings to your applications.
+- [Users and Roles](../users-roles.md): Configure authentication and authorization mechanisms, built-in users, external providers, and more.
+- [Autoscaling](../autoscaling.md): Learn how to use {{es}} autoscaling on ECK, or use Horizontal Pod Autoscaler functionality for stateless workloads.
+- [Licensing](../license/manage-your-license-in-eck.md): Manage licenses on ECK.
+
+::::{important}
+ECK is an Elastic self-managed product offered in two licensing tiers: Basic and Enterprise. For more details refer to [Elastic subscriptions](https://www.elastic.co/subscriptions) and [](/deploy-manage/license/manage-your-license-in-eck.md) documentation.
+::::
+
+## Quickstart [eck-quickstart]
+
+If you want to get started quickly, follow these guides to deploy ECK and set up an {{es}} cluster:
+
+* [Install ECK using YAML manifests](./cloud-on-k8s/install-using-yaml-manifest-quickstart.md)
+* [Deploy an {{es}} cluster](./cloud-on-k8s/elasticsearch-deployment-quickstart.md)
+* [Deploy a {{kib}} instance](./cloud-on-k8s/kibana-instance-quickstart.md)
+
+Afterwards, you can:
+
+* Learn how to [update your deployment](./cloud-on-k8s/update-deployments.md)
+* Check out [our recipes](./cloud-on-k8s/recipes.md) for multiple use cases
+* Find further sample resources [in the project repository](https://github.com/elastic/cloud-on-k8s/tree/2.16/config/samples)
+
+## Supported versions [k8s-supported]
+
+This section outlines the supported Kubernetes and Elastic Stack versions for ECK. Check the full [Elastic support matrix](https://www.elastic.co/support/matrix#matrix_kubernetes) for more information.
+
+### Kubernetes compatibility
+
+ECK is compatible with the following Kubernetes distributions and related technologies:
+
+* Kubernetes 1.28-1.32
+* OpenShift 4.12-4.17
+* Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), and Amazon Elastic Kubernetes Service (EKS)
+* Helm: 3.2.0+
+
+ECK should work with all conformant **installers** listed in these [FAQs](https://github.com/cncf/k8s-conformance/blob/master/faq.md#what-is-a-distribution-hosted-platform-and-an-installer). Distributions include source patches and so may not work as-is with ECK.
+
+### Elastic Stack compatibility
+
+ECK is compatible with the following Elastic Stack applications:
+
+* Elasticsearch, Kibana, APM Server: 6.8+, 7.1+, 8+
+* Enterprise Search: 7.7+, 8+
+* Beats: 7.0+, 8+
+* Elastic Agent: 7.10+ (standalone), 7.14+ (Fleet), 8+
+* Elastic Maps Server: 7.11+, 8+
+* Logstash: 8.7+
+
+Elastic Stack application images for the OpenShift-certified Elasticsearch (ECK) Operator are only available from version 7.10 and later.
+
+## Learn more about ECK [k8s_learn_more_about_eck]
+
+* [Orchestrate Elasticsearch on Kubernetes](https://www.elastic.co/elasticsearch-kubernetes)
+* [ECK post on the Elastic Blog](https://www.elastic.co/blog/introducing-elastic-cloud-on-kubernetes-the-elasticsearch-operator-and-beyond?elektra=products&storm=sub1)
+* [Getting Started With Elastic Cloud on Kubernetes (ECK)](https://www.youtube.com/watch?v=PIJmlYBIFXM)
+* [Running the Elastic Stack on Kubernetes with ECK](https://www.youtube.com/watch?v=Wf6E3vkvEFM)

deploy-manage/deploy/cloud-on-k8s/accessing-services.md

@@ -1,28 +1,133 @@
 ---
+applies:
+  eck: all
 mapped_urls:
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-accessing-elastic-services.html
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-request-elasticsearch-endpoint.html
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-services.html
-  - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-security.html
 ---
 
-# Accessing services
+# Accessing services [k8s-accessing-elastic-services]
 
-% What needs to be done: Refine
+To provide access to {{es}}, {{kib}}, and other {{stack}} applications when applicable, ECK relies on [Kubernetes services](https://kubernetes.io/docs/concepts/services-networking/service/).
 
-% GitHub issue: https://github.com/elastic/docs-projects/issues/357
+All Elastic Stack resources deployed by the ECK operator are secured by default. The operator sets up basic authentication and TLS to encrypt network traffic to, from, and within your Elasticsearch cluster.
 
-% Scope notes: Merge the selected docs into one: - First describe how to access Elasticsearch. - Describe the services that ECK creates for ES. - Provide the example and instructions
+This section explains how to access and customize the Kubernetes services and secrets created by ECK, covering topics such as:
 
-% Use migrated content from existing pages that map to this page:
+* [Retrieving the `elastic` user password for basic authentication](#k8s-authentication)
+* [Managing Kubernetes services](#k8s-kubernetes-service)
+* [Obtaining the CA certificate and accessing the endpoint](#k8s-request-elasticsearch-endpoint)
 
-% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md
-% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md
-% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-services.md
-% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-security.md
+For advanced use cases related to exposing and accessing orchestrated applications, see:
+
+* [](/deploy-manage/security/secure-http-communications.md): Configuration options for the HTTP SSL certificates, including integration with certificate management systems such as [cert-manager](https://cert-manager.io/).
+* [](./service-meshes.md): Connect ECK and your managed deployments to service mesh implementations such as [Istio](https://istio.io) and [Linkerd](https://linkerd.io).
+* [](./requests-routing-to-elasticsearch-nodes.md): Create custom services to expose different node types.
+* [Use Ingress to expose {{es}} or {{kib}}](./managing-deployments-using-helm-chart.md#k8s-eck-stack-ingress): Helm based installation also facilitates the creation of Ingress resources.
+
+## Retrieve the `elastic` user password [k8s-authentication]
+
+To access Elastic resources, the operator manages a default user named `elastic` with the `superuser` role. Its password is stored in a `Secret` named `<name>-elastic-user`.
+
+Run the following command to obtain the password of the `elastic` user:
+
+```sh
+> kubectl get secret hulk-es-elastic-user -o go-template='{{.data.elastic | base64decode }}'
+42xyz42citsale42xyz42
+```
+
+::::{note}
+Beware of copying this Secret as-is into a different namespace. Check [Common Problems: Owner References](../../../troubleshoot/deployments/cloud-on-k8s/common-problems.md#k8s-common-problems-owner-refs) for more information.
+::::
+
+For more information about handling built-in users on ECK deployments, refer to [](/deploy-manage/users-roles/cluster-or-deployment-auth/built-in-users.md).
+
+## Managing Kubernetes services [k8s-kubernetes-service]
+
+You can access Elastic resources by using native Kubernetes services that are not reachable from the public Internet by default.
+
+For each resource, the operator manages a Kubernetes service named `<name>-[es|kb|apm|ent|agent]-http`, which is of type `ClusterIP` by default. `ClusterIP` exposes the service on a cluster-internal IP and makes the service only reachable within the cluster.
+
+```sh
+> kubectl get svc
+
+NAME                TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)          AGE
+hulk-apm-http       ClusterIP      10.19.212.105   <none>           8200/TCP   1m
+hulk-es-http        ClusterIP      10.19.252.160   <none>           9200/TCP   1m
+hulk-kb-http        ClusterIP      10.19.247.151   <none>           5601/TCP   1m
+```
+
+### Allow public access [k8s-allow-public-access]
+
+You can expose services in [different ways](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types) by specifying an `http.service.spec.type` in the `spec` of the resource manifest. On cloud providers which support external load balancers, you can set the `type` field to `LoadBalancer` to provision a load balancer for the `Service`, and populate the column `EXTERNAL-IP` after a short delay. Depending on the cloud provider, it may incur costs.
+
+By default, the Elasticsearch service created by ECK is configured to route traffic to all Elasticsearch nodes in the cluster. Depending on your cluster configuration, you may want more control over the set of nodes that handle different types of traffic (query, ingest, and so on). Refer to [](./requests-routing-to-elasticsearch-nodes.md) for more information.
+
+::::{warning}
+When you change the `clusterIP` setting of the service, ECK will delete and re-create the service as `clusterIP` is an immutable field. Depending on your client implementation, this might result in a short disruption until the service DNS entries refresh to point to the new endpoints.
+::::
+
+```yaml
+apiVersion: <kind>.k8s.elastic.co/v1
+kind: <Kind>
+metadata:
+  name: hulk
+spec:
+  version: 8.16.1
+  http:
+    service:
+      spec:
+        type: LoadBalancer
+```
+
+```sh
+> kubectl get svc
+
+NAME                TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)          AGE
+hulk-apm-http       LoadBalancer   10.19.212.105   35.176.227.106   8200:31000/TCP   1m
+hulk-es-http        LoadBalancer   10.19.252.160   35.198.131.115   9200:31320/TCP   1m
+hulk-kb-http        LoadBalancer   10.19.247.151   35.242.197.228   5601:31380/TCP   1m
+```
+
+## Access the endpoint [k8s-request-elasticsearch-endpoint]
+
+You can access the Elasticsearch endpoint within or outside the Kubernetes cluster.
+
+**Within the Kubernetes cluster**
+
+1. Retrieve the CA certificate.
+2. Retrieve the password of the `elastic` user.
+3. Use the service name to access the endpoint.
+
+```sh
+NAME=hulk
+
+kubectl get secret "$NAME-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt
+PW=$(kubectl get secret "$NAME-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}')
+
+curl --cacert tls.crt -u elastic:$PW https://$NAME-es-http:9200/
+```
+
+::::{tip}
+You can also use the examples in this section to access {{kib}} instead of {{es}} by adapting the secret and service names.
+::::
+
+**Outside the Kubernetes cluster**
+
+1. Retrieve the CA certificate.
+2. Retrieve the password of the `elastic` user.
+3. Retrieve the IP of the `LoadBalancer` service.
+
+```sh
+NAME=hulk
+
+kubectl get secret "$NAME-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt
+IP=$(kubectl get svc "$NAME-es-http" -o jsonpath='{.status.loadBalancer.ingress[].ip}')
+PW=$(kubectl get secret "$NAME-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}')
+
+curl --cacert tls.crt -u elastic:$PW https://$IP:9200/
+```
 
-% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc):
 
-$$$k8s-allow-public-access$$$
 
-$$$k8s-setting-up-your-own-certificate$$$

deploy-manage/deploy/cloud-on-k8s/advanced-configuration-logstash.md

@@ -1,4 +1,6 @@
 ---
+applies:
+  eck: all
 mapped_pages:
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-advanced-configuration.html
 ---

deploy-manage/deploy/cloud-on-k8s/advanced-configuration-maps-server.md

@@ -1,4 +1,6 @@
 ---
+applies:
+  eck: all
 mapped_pages:
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-maps-advanced-configuration.html
 ---

deploy-manage/deploy/cloud-on-k8s/advanced-configuration.md

@@ -1,4 +1,6 @@
 ---
+applies:
+  eck: all
 mapped_pages:
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-advanced-configuration.html
 ---
@@ -166,8 +168,6 @@ Now that you know how to use the APM keystore and customize the server configura
 
 By default the operator manages a private CA and generates a self-signed certificate used to secure the communication between APM agents and the server.
 
-This behavior and the relevant configuration is identical to what is done for Elasticsearch and Kibana. Check [Setting up your own certificate](accessing-services.md#k8s-setting-up-your-own-certificate) for more information on how to use your own certificate to configure the TLS endpoint of the APM Server.
+This behavior and the relevant configuration is identical to what is done for Elasticsearch and Kibana. Check [Setting up your own certificate](/deploy-manage/security/secure-http-communications.md) for more information on how to use your own certificate to configure the TLS endpoint of the APM Server.
 
 For more details on how to configure the APM agents to work with custom certificates, check the [APM agents documentation](https://www.elastic.co/guide/en/apm/agent/index.html).
-
-

deploy-manage/deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md

@@ -1,4 +1,6 @@
 ---
+applies:
+  eck: all
 mapped_pages:
   - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-node-scheduling.html
 ---