diff --git a/deploy-manage/deploy/cloud-on-k8s.md b/deploy-manage/deploy/cloud-on-k8s.md index dff4d8ffcc..46972c3211 100644 --- a/deploy-manage/deploy/cloud-on-k8s.md +++ b/deploy-manage/deploy/cloud-on-k8s.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_urls: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-overview.html - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-topics.html @@ -6,18 +8,86 @@ mapped_urls: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s_learn_more_about_eck.html --- -# Elastic Cloud on Kubernetes +# Elastic Cloud on Kubernetes [k8s-overview] -% What needs to be done: Refine +Built on the Kubernetes Operator pattern, {{eck}} (ECK) extends the basic Kubernetes orchestration capabilities to support the setup and management of Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash on Kubernetes. -% GitHub issue: https://github.com/elastic/docs-projects/issues/357 +## ECK overview -% Scope notes: Maybe we can even leave it as it is. +With Elastic Cloud on Kubernetes, you can streamline critical operations, such as: -% Use migrated content from existing pages that map to this page: +1. Managing and monitoring multiple clusters +2. Scaling cluster capacity and storage +3. Performing safe configuration changes through rolling upgrades +4. Securing clusters with TLS certificates +5. Setting up hot-warm-cold architectures with availability zone awareness -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-overview.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md -% Notes: redirect only -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-supported.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md \ No newline at end of file +This section provides everything you need to install, configure, and manage Elastic Stack applications with ECK, including: + +- [](./cloud-on-k8s/deploy-an-orchestrator.md): ECK installation methods and configuration options. Deploy ECK on managed Kubernetes platforms like GKE, AKS, and EKS, on self-managed Kubernetes clusters, on OpenShift, and even in air-gapped environments. +- [](./cloud-on-k8s/manage-deployments.md): Handle {{es}} clusters and {{kib}} instances through ECK. +- [](./cloud-on-k8s/orchestrate-other-elastic-applications.md): Run APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash on Kubernetes. +- [](./cloud-on-k8s/tools-apis.md): A collection of tools and APIs available in ECK based environments. + +Other sections of the documentation include the following important topics around ECK: + +- [Logging and Monitoring](../monitor.md): Configure stack monitoring and logs forwarding with the help of ECK. +- [Remote Clusters](../remote-clusters.md): Configure remote clusters on ECK. +- [](../tools.md): Add snapshot repositories to your {{es}} clusters for automatic snapshots. +- [Security](../security.md): Secure communications, manage HTTP certificates, or add secure settings to your applications. +- [Users and Roles](../users-roles.md): Configure authentication and authorization mechanisms, built-in users, external providers, and more. +- [Autoscaling](../autoscaling.md): Learn how to use {{es}} autoscaling on ECK, or use Horizontal Pod Autoscaler functionality for stateless workloads. +- [Licensing](../license/manage-your-license-in-eck.md): Manage licenses on ECK. + +::::{important} +ECK is an Elastic self-managed product offered in two licensing tiers: Basic and Enterprise. For more details refer to [Elastic subscriptions](https://www.elastic.co/subscriptions) and [](/deploy-manage/license/manage-your-license-in-eck.md) documentation. +:::: + +## Quickstart [eck-quickstart] + +If you want to get started quickly, follow these guides to deploy ECK and set up an {{es}} cluster: + +* [Install ECK using YAML manifests](./cloud-on-k8s/install-using-yaml-manifest-quickstart.md) +* [Deploy an {{es}} cluster](./cloud-on-k8s/elasticsearch-deployment-quickstart.md) +* [Deploy a {{kib}} instance](./cloud-on-k8s/kibana-instance-quickstart.md) + +Afterwards, you can: + +* Learn how to [update your deployment](./cloud-on-k8s/update-deployments.md) +* Check out [our recipes](./cloud-on-k8s/recipes.md) for multiple use cases +* Find further sample resources [in the project repository](https://github.com/elastic/cloud-on-k8s/tree/2.16/config/samples) + +## Supported versions [k8s-supported] + +This section outlines the supported Kubernetes and Elastic Stack versions for ECK. Check the full [Elastic support matrix](https://www.elastic.co/support/matrix#matrix_kubernetes) for more information. + +### Kubernetes compatibility + +ECK is compatible with the following Kubernetes distributions and related technologies: + +* Kubernetes 1.28-1.32 +* OpenShift 4.12-4.17 +* Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), and Amazon Elastic Kubernetes Service (EKS) +* Helm: 3.2.0+ + +ECK should work with all conformant **installers** listed in these [FAQs](https://github.com/cncf/k8s-conformance/blob/master/faq.md#what-is-a-distribution-hosted-platform-and-an-installer). Distributions include source patches and so may not work as-is with ECK. + +### Elastic Stack compatibility + +ECK is compatible with the following Elastic Stack applications: + +* Elasticsearch, Kibana, APM Server: 6.8+, 7.1+, 8+ +* Enterprise Search: 7.7+, 8+ +* Beats: 7.0+, 8+ +* Elastic Agent: 7.10+ (standalone), 7.14+ (Fleet), 8+ +* Elastic Maps Server: 7.11+, 8+ +* Logstash: 8.7+ + +Elastic Stack application images for the OpenShift-certified Elasticsearch (ECK) Operator are only available from version 7.10 and later. + +## Learn more about ECK [k8s_learn_more_about_eck] + +* [Orchestrate Elasticsearch on Kubernetes](https://www.elastic.co/elasticsearch-kubernetes) +* [ECK post on the Elastic Blog](https://www.elastic.co/blog/introducing-elastic-cloud-on-kubernetes-the-elasticsearch-operator-and-beyond?elektra=products&storm=sub1) +* [Getting Started With Elastic Cloud on Kubernetes (ECK)](https://www.youtube.com/watch?v=PIJmlYBIFXM) +* [Running the Elastic Stack on Kubernetes with ECK](https://www.youtube.com/watch?v=Wf6E3vkvEFM) diff --git a/deploy-manage/deploy/cloud-on-k8s/accessing-services.md b/deploy-manage/deploy/cloud-on-k8s/accessing-services.md index 7b6d059eef..deb0a80869 100644 --- a/deploy-manage/deploy/cloud-on-k8s/accessing-services.md +++ b/deploy-manage/deploy/cloud-on-k8s/accessing-services.md @@ -1,28 +1,133 @@ --- +applies: + eck: all mapped_urls: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-accessing-elastic-services.html - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-request-elasticsearch-endpoint.html - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-services.html - - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-security.html --- -# Accessing services +# Accessing services [k8s-accessing-elastic-services] -% What needs to be done: Refine +To provide access to {{es}}, {{kib}}, and other {{stack}} applications when applicable, ECK relies on [Kubernetes services](https://kubernetes.io/docs/concepts/services-networking/service/). -% GitHub issue: https://github.com/elastic/docs-projects/issues/357 +All Elastic Stack resources deployed by the ECK operator are secured by default. The operator sets up basic authentication and TLS to encrypt network traffic to, from, and within your Elasticsearch cluster. -% Scope notes: Merge the selected docs into one: - First describe how to access Elasticsearch. - Describe the services that ECK creates for ES. - Provide the example and instructions +This section explains how to access and customize the Kubernetes services and secrets created by ECK, covering topics such as: -% Use migrated content from existing pages that map to this page: +* [Retrieving the `elastic` user password for basic authentication](#k8s-authentication) +* [Managing Kubernetes services](#k8s-kubernetes-service) +* [Obtaining the CA certificate and accessing the endpoint](#k8s-request-elasticsearch-endpoint) -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-services.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-security.md +For advanced use cases related to exposing and accessing orchestrated applications, see: + +* [](/deploy-manage/security/secure-http-communications.md): Configuration options for the HTTP SSL certificates, including integration with certificate management systems such as [cert-manager](https://cert-manager.io/). +* [](./service-meshes.md): Connect ECK and your managed deployments to service mesh implementations such as [Istio](https://istio.io) and [Linkerd](https://linkerd.io). +* [](./requests-routing-to-elasticsearch-nodes.md): Create custom services to expose different node types. +* [Use Ingress to expose {{es}} or {{kib}}](./managing-deployments-using-helm-chart.md#k8s-eck-stack-ingress): Helm based installation also facilitates the creation of Ingress resources. + +## Retrieve the `elastic` user password [k8s-authentication] + +To access Elastic resources, the operator manages a default user named `elastic` with the `superuser` role. Its password is stored in a `Secret` named `-elastic-user`. + +Run the following command to obtain the password of the `elastic` user: + +```sh +> kubectl get secret hulk-es-elastic-user -o go-template='{{.data.elastic | base64decode }}' +42xyz42citsale42xyz42 +``` + +::::{note} +Beware of copying this Secret as-is into a different namespace. Check [Common Problems: Owner References](../../../troubleshoot/deployments/cloud-on-k8s/common-problems.md#k8s-common-problems-owner-refs) for more information. +:::: + +For more information about handling built-in users on ECK deployments, refer to [](/deploy-manage/users-roles/cluster-or-deployment-auth/built-in-users.md). + +## Managing Kubernetes services [k8s-kubernetes-service] + +You can access Elastic resources by using native Kubernetes services that are not reachable from the public Internet by default. + +For each resource, the operator manages a Kubernetes service named `-[es|kb|apm|ent|agent]-http`, which is of type `ClusterIP` by default. `ClusterIP` exposes the service on a cluster-internal IP and makes the service only reachable within the cluster. + +```sh +> kubectl get svc + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +hulk-apm-http ClusterIP 10.19.212.105 8200/TCP 1m +hulk-es-http ClusterIP 10.19.252.160 9200/TCP 1m +hulk-kb-http ClusterIP 10.19.247.151 5601/TCP 1m +``` + +### Allow public access [k8s-allow-public-access] + +You can expose services in [different ways](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types) by specifying an `http.service.spec.type` in the `spec` of the resource manifest. On cloud providers which support external load balancers, you can set the `type` field to `LoadBalancer` to provision a load balancer for the `Service`, and populate the column `EXTERNAL-IP` after a short delay. Depending on the cloud provider, it may incur costs. + +By default, the Elasticsearch service created by ECK is configured to route traffic to all Elasticsearch nodes in the cluster. Depending on your cluster configuration, you may want more control over the set of nodes that handle different types of traffic (query, ingest, and so on). Refer to [](./requests-routing-to-elasticsearch-nodes.md) for more information. + +::::{warning} +When you change the `clusterIP` setting of the service, ECK will delete and re-create the service as `clusterIP` is an immutable field. Depending on your client implementation, this might result in a short disruption until the service DNS entries refresh to point to the new endpoints. +:::: + +```yaml +apiVersion: .k8s.elastic.co/v1 +kind: +metadata: + name: hulk +spec: + version: 8.16.1 + http: + service: + spec: + type: LoadBalancer +``` + +```sh +> kubectl get svc + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +hulk-apm-http LoadBalancer 10.19.212.105 35.176.227.106 8200:31000/TCP 1m +hulk-es-http LoadBalancer 10.19.252.160 35.198.131.115 9200:31320/TCP 1m +hulk-kb-http LoadBalancer 10.19.247.151 35.242.197.228 5601:31380/TCP 1m +``` + +## Access the endpoint [k8s-request-elasticsearch-endpoint] + +You can access the Elasticsearch endpoint within or outside the Kubernetes cluster. + +**Within the Kubernetes cluster** + +1. Retrieve the CA certificate. +2. Retrieve the password of the `elastic` user. +3. Use the service name to access the endpoint. + +```sh +NAME=hulk + +kubectl get secret "$NAME-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt +PW=$(kubectl get secret "$NAME-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}') + +curl --cacert tls.crt -u elastic:$PW https://$NAME-es-http:9200/ +``` + +::::{tip} +You can also use the examples in this section to access {{kib}} instead of {{es}} by adapting the secret and service names. +:::: + +**Outside the Kubernetes cluster** + +1. Retrieve the CA certificate. +2. Retrieve the password of the `elastic` user. +3. Retrieve the IP of the `LoadBalancer` service. + +```sh +NAME=hulk + +kubectl get secret "$NAME-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt +IP=$(kubectl get svc "$NAME-es-http" -o jsonpath='{.status.loadBalancer.ingress[].ip}') +PW=$(kubectl get secret "$NAME-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}') + +curl --cacert tls.crt -u elastic:$PW https://$IP:9200/ +``` -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): -$$$k8s-allow-public-access$$$ -$$$k8s-setting-up-your-own-certificate$$$ \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-logstash.md b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-logstash.md index 6972733ee9..2d0183b1c0 100644 --- a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-advanced-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-maps-server.md b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-maps-server.md index 9478d3d322..f789eaee80 100644 --- a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-maps-server.md +++ b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-maps-server.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-maps-advanced-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration.md b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration.md index e69e1d0faa..3390f9e231 100644 --- a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration.md +++ b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-advanced-configuration.html --- @@ -166,8 +168,6 @@ Now that you know how to use the APM keystore and customize the server configura By default the operator manages a private CA and generates a self-signed certificate used to secure the communication between APM agents and the server. -This behavior and the relevant configuration is identical to what is done for Elasticsearch and Kibana. Check [Setting up your own certificate](accessing-services.md#k8s-setting-up-your-own-certificate) for more information on how to use your own certificate to configure the TLS endpoint of the APM Server. +This behavior and the relevant configuration is identical to what is done for Elasticsearch and Kibana. Check [Setting up your own certificate](/deploy-manage/security/secure-http-communications.md) for more information on how to use your own certificate to configure the TLS endpoint of the APM Server. For more details on how to configure the APM agents to work with custom certificates, check the [APM agents documentation](https://www.elastic.co/guide/en/apm/agent/index.html). - - diff --git a/deploy-manage/deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md b/deploy-manage/deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md index 02dfffb5e1..f7993f3580 100644 --- a/deploy-manage/deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md +++ b/deploy-manage/deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-node-scheduling.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/air-gapped-install.md b/deploy-manage/deploy/cloud-on-k8s/air-gapped-install.md index 0fae72fba5..8d1e878aa6 100644 --- a/deploy-manage/deploy/cloud-on-k8s/air-gapped-install.md +++ b/deploy-manage/deploy/cloud-on-k8s/air-gapped-install.md @@ -1,4 +1,7 @@ --- +navigation_title: Air gapped environments +applies: + eck: all mapped_urls: - https://www.elastic.co/guide/en/elastic-stack/current/air-gapped-install.html - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-air-gapped.html @@ -15,6 +18,8 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/stack-docs/elastic-stack/air-gapped-install.md + +% already removed % - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-air-gapped.md % Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): @@ -24,3 +29,132 @@ $$$air-gapped-install$$$ $$$k8s-container-registry-override$$$ $$$k8s-eck-diag-air-gapped$$$ + +% There are two concepts and areas to explore here: +% ECK installation on air-gapped. This has no complexity as it's all a matter of docker registry and docker images. +% Managing deployments on an ECK running on air-gapped is something not really covered in the official ECK book and partly covered in stack-docs + +% In this doc we will focus on ECK operator installation in air gapped environments, and we will link to Manage Deployments -> Air gapped (doesn't exist yet) for the content and examples about the rest. + +% from fleet air-gapped +% Kibana is able to reach the Elastic Package Registry to download package metadata and content. +% Elastic Agents are able to download binaries during upgrades from the Elastic Artifact Registry. + +% what about Elasticsearch requirements for example for GeoIP database, etc? + +Pending to determine what to do with this: +* Syncing container images for ECK and all other {{stack}} components over to a locally-accessible container repository. +* Modifying the ECK helm chart configuration so that ECK is aware that it is supposed to use your offline container repository instead of the public Elastic repository. +* Optionally, disabling ECK telemetry collection in the ECK helm chart. This configuration propagates to all other Elastic components, such as {{kib}}. +* Building your custom deployment container image for the {{artifact-registry}}. +* Building your custom deployment container image for the Elastic Endpoint Artifact Repository. + +# Running in air-gapped environments [k8s-air-gapped] + +The ECK operator can be run in an air-gapped environment without access to the open internet when it is configured not to pull container images from `docker.elastic.co`. + +By default ECK does not require you to specify the container image for each Elastic Stack application you deploy. + +```yaml +apiVersion: elasticsearch.k8s.elastic.co/v1 +kind: Elasticsearch +metadata: + name: quickstart +spec: + version: 8.16.1 + # image: docker.elastic.co/elasticsearch/elasticsearch:8.16.1 <1> + nodeSets: + - name: default + count: 1 + # podTemplate: + # spec: + # imagePullSecrets: <2> + # - name: private-registry-credentials-secret +``` + +1. The ECK operator will set this value by default. You can explicitly set it to your mirrored container image when running in an air-gapped environment +2. You can provide credentials to your private container registry by setting the `imagePullSecrets` field through the `spec.podTemplate` section of your Elastic resource specification, check [how to customize the Elastic resources Pods](../../../deploy-manage/deploy/cloud-on-k8s/customize-pods.md) and [how to setup a Secret containing your registry credentials](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). + + +ECK will automatically set the correct container image for each application. When running in an air-gapped or offline environment you will have to mirror the official Elastic container images in a private container image registry. To make use of your mirrored images you can either set the image for each application explicitly as shown in the preceding example or more conveniently override the default container registry as explained in the next section. + + +## Use a mirrored image of the ECK operator [k8s-use-mirrored-operator-image] + +To deploy the ECK operator in an air-gapped environment, you first have to mirror the operator image itself from `docker.elastic.co` to a private container registry, for example `my.registry`. + +Once the ECK operator image is copied internally, replace the original image name `docker.elastic.co/eck/eck-operator:2.16.1` with the private name of the image, for example `my.registry/eck/eck-operator:2.16.1`, in the [operator manifests](../../../deploy-manage/deploy/cloud-on-k8s/install-using-yaml-manifest-quickstart.md). When using [Helm charts](../../../deploy-manage/deploy/cloud-on-k8s/install-using-helm-chart.md), replace the `image.repository` Helm value with, for example, `my.registry/eck/eck-operator`. + + +## Override the default container registry [k8s-container-registry-override] + +When creating custom resources (Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash), the operator defaults to using container images pulled from the `docker.elastic.co` registry. If you are in an environment where external network access is restricted, you can configure the operator to use a different default container registry by starting the operator with the `--container-registry` command-line flag. Check [*Configure ECK*](../../../deploy-manage/deploy/cloud-on-k8s/configure-eck.md) for more information on how to configure the operator using command-line flags and environment variables. + +The operator expects container images to be located at specific repositories in the default container registry. Make sure that your container images are stored in the right repositories and are tagged correctly with the Stack version number. For example, if your private registry is `my.registry` and you wish to deploy components from Stack version 8.16.1, the following image names should exist: + +* `my.registry/elasticsearch/elasticsearch:8.16.1` +* `my.registry/kibana/kibana:8.16.1` +* `my.registry/apm/apm-server:8.16.1` + + +## Use a global container repository [k8s-container-repository-override] + +If you cannot follow the default Elastic image repositories naming scheme, you can configure the operator to use a different container repository by starting the operator with the `--container-repository` command-line flag. Check [*Configure ECK*](../../../deploy-manage/deploy/cloud-on-k8s/configure-eck.md) for more information on how to configure the operator using command-line flags and environment variables. + +For example, if your private registry is `my.registry` and all Elastic images are located under the `elastic` repository, the following image names should exist: + +* `my.registry/elastic/elasticsearch:8.16.1` +* `my.registry/elastic/kibana:8.16.1` +* `my.registry/elastic/apm-server:8.16.1` + + +## ECK Diagnostics in air-gapped environments [k8s-eck-diag-air-gapped] + +The [eck-diagnostics tool](../../../troubleshoot/deployments/cloud-on-k8s/run-eck-diagnostics.md) optionally runs diagnostics for Elastic Stack applications in a separate container that is deployed into the Kubernetes cluster. + +In air-gapped environments with no access to the `docker.elastic.co` registry, you should copy the latest support-diagnostics container image to your internal image registry and then run the tool with the additional flag `--diagnostic-image `. To find out which support diagnostics container image matches your version of eck-diagnostics run the tool once without arguments and it will print the default image in use. + + +% FROM THE OTHER CONTENT (ELASTIC-STACK): + +### 2. Kubernetes & OpenShift Install [air-gapped-kubernetes-and-openshift] + +Setting up air-gapped Kubernetes or OpenShift installs of the {{stack}} has some unique concerns, but the general dependencies are the same as in the self-managed install case on a regular Linux machine. + + +#### 2.1. Elastic Kubernetes Operator (ECK) [air-gapped-k8s-os-elastic-kubernetes-operator] + +The Elastic Kubernetes operator is an additional component in the Kubernetes OpenShift install that, essentially, does a lot of the work in installing, configuring, and updating deployments of the {{stack}}. For details, refer to the [{{eck}} install instructions](../../../deploy-manage/deploy/cloud-on-k8s/air-gapped-install.md). + +The main requirements are: + +* Syncing container images for ECK and all other {{stack}} components over to a locally-accessible container repository. +* Modifying the ECK helm chart configuration so that ECK is aware that it is supposed to use your offline container repository instead of the public Elastic repository. +* Optionally, disabling ECK telemetry collection in the ECK helm chart. This configuration propagates to all other Elastic components, such as {{kib}}. +* Building your custom deployment container image for the {{artifact-registry}}. +* Building your custom deployment container image for the Elastic Endpoint Artifact Repository. + + +#### 2.2. Elastic Package Registry [air-gapped-k8s-os-elastic-package-registry] + +The container image can be downloaded from the official Elastic Docker repository, as described in the {{fleet}} and {{elastic-agent}} [air-gapped environments](https://www.elastic.co/guide/en/fleet/current/air-gapped.html) documentation. + +This container would, ideally, run as a Kubernetes deployment. Refer to [Appendix C - EPR Kubernetes Deployment](../../../deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-epr-kubernetes-example) for examples. + + +#### 2.3. {{artifact-registry}} [air-gapped-k8s-os-elastic-artifact-registry] + +A custom container would need to be created following similar instructions to setting up a web server in the [self-managed install case](../../../deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-artifact-registry). For example, a container file using an NGINX base image could be used to run a build similar to the example described in [Appendix B - {{artifact-registry}}](../../../deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-artifact-registry-example). + + +#### 2.4. Elastic Endpoint Artifact Repository [air-gapped-k8s-os-elastic-endpoint-artifact-repository] + +Just like the {{artifact-registry}}. A custom container needs to be created following similar instructions to setting up a web server for the [self-managed install case](../../../deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-artifact-registry). + + +#### 2.5. Ironbank Secure Images for Elastic [air-gapped-k8s-os-ironbank-secure-images] + +Besides the public [Elastic container repository](https://www.docker.elastic.co), most {{stack}} container images are also available in Platform One’s [Iron Bank](https://ironbank.dso.mil/repomap?vendorFilters=Elastic&page=1&sort=1). + + + diff --git a/deploy-manage/deploy/cloud-on-k8s/apm-server.md b/deploy-manage/deploy/cloud-on-k8s/apm-server.md index 8a9f9144e1..288d73905c 100644 --- a/deploy-manage/deploy/cloud-on-k8s/apm-server.md +++ b/deploy-manage/deploy/cloud-on-k8s/apm-server.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-server.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/beats.md b/deploy-manage/deploy/cloud-on-k8s/beats.md index 25f3de6c50..88685012e1 100644 --- a/deploy-manage/deploy/cloud-on-k8s/beats.md +++ b/deploy-manage/deploy/cloud-on-k8s/beats.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-beats.md b/deploy-manage/deploy/cloud-on-k8s/configuration-beats.md index ffb89ba39d..216150127a 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-beats.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-beats.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-beats.md b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-beats.md index ae4c75789a..eaa702b993 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-beats.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-beats.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat-configuration-examples.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-fleet.md b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-fleet.md index 9cea9876e7..ef004f2946 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-fleet.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-fleet.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-fleet-configuration-examples.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-logstash.md b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-logstash.md index 1d7ca54ae4..9640071f3c 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-configuration-examples.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-standalone.md b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-standalone.md index 73f6d7f125..24a0312025 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-standalone.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-standalone.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-configuration-examples.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md b/deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md index c953f78d2a..b7760578bb 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-fleet-configuration.html --- @@ -221,7 +223,7 @@ To deploy {{agent}} in clusters with the Pod Security Policy admission controlle ## Customize {{fleet-server}} Service [k8s-elastic-agent-fleet-configuration-customize-fleet-server-service] -By default, ECK creates a Service for {{fleet-server}} that {{agents}} can connect through. You can customize it using the `http` configuration element. Check more information on how to [make changes](accessing-services.md) to the Service and [customize](tls-certificates.md) the TLS configuration. +By default, ECK creates a Service for {{fleet-server}} that {{agents}} can connect through. You can customize it using the `http` configuration element. Check more information on how to [make changes](accessing-services.md) to the Service and [customize](/deploy-manage/security/secure-http-communications.md) the TLS configuration. ## Control {{fleet}} policy selection [k8s-elastic-agent-control-fleet-policy-selection] diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-logstash.md b/deploy-manage/deploy/cloud-on-k8s/configuration-logstash.md index 87d1c9f185..eaa6135de0 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-standalone.md b/deploy-manage/deploy/cloud-on-k8s/configuration-standalone.md index 0c2516d2c9..f05f3b8bf2 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-standalone.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-standalone.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configure-deployments.md b/deploy-manage/deploy/cloud-on-k8s/configure-deployments.md index 857a783f20..ec4294b640 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configure-deployments.md +++ b/deploy-manage/deploy/cloud-on-k8s/configure-deployments.md @@ -1,26 +1,35 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-orchestrating-elastic-stack-applications.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-update-deployment.html --- # Configure deployments [k8s-orchestrating-elastic-stack-applications] -* [*Run Elasticsearch on ECK*](elasticsearch-configuration.md) -* [*Run {{kib}} on ECK*](kibana-configuration.md) -* [*Run APM Server on ECK*](apm-server.md) -* [*Run standalone Elastic Agent on ECK*](standalone-elastic-agent.md) -* [*Run {{fleet}}-managed {{agent}} on ECK*](fleet-managed-elastic-agent.md) -* [*Run Elastic Maps Server on ECK*](elastic-maps-server.md) -* [*Run Beats on ECK*](beats.md) -* [*Run {{ls}} on ECK*](logstash.md) -* [*Elastic Stack Helm Chart*](managing-deployments-using-helm-chart.md) -* [*Recipes*](recipes.md) -* [*Secure the Elastic Stack*](../../security.md) -* [*Access Elastic Stack services*](accessing-services.md) -* [*Customize Pods*](customize-pods.md) -* [*Manage compute resources*](manage-compute-resources.md) -* [*Autoscaling stateless applications*](../../autoscaling/autoscaling-stateless-applications-on-eck.md) -* [*Elastic Stack configuration policies*](elastic-stack-configuration-policies.md) -* [*Upgrade the Elastic Stack version*](../../upgrade/deployment-or-cluster.md) -* [*Connect to external Elastic resources*](connect-to-external-elastic-resources.md) +This section provides details around {{kib}} and {{es}} configuration when running on ECK. For general information about how ECK applies configuration changes and the syntax to use in the YAML manifests, refer to [](./update-deployments.md). +* [**{{es}} configuration**](elasticsearch-configuration.md): Review configuration possibilities to tune your {{es}} cluster running on ECK, learn how [nodes orchestration](./nodes-orchestration.md) work, [storage recommendations](./storage-recommendations.md), and more. + +* [**{{kib}} configuration**](kibana-configuration.md): Learn how to connect {{kib}} to an {{es}} cluster, apply advanced configuration settings, and tune the HTTP configuration. + +Additionally, the following topics apply to both {{es}} and {{kib}}, and in some cases, to other applications supported by ECK: + +* [**Access services**](accessing-services.md): Learn how to access to the orchestrated clusters and how to adapt the Kubernetes services to your needs. + +* [**Customize Pods**](customize-pods.md): Learn how to adapt the `podTemplate` field to your needs. + +* [**Manage compute resources**](manage-compute-resources.md): Important considerations around CPU and memory `requests` and `limits` when running production workloads. + +* [**Recipes**](recipes.md): Advanced use cases examples available in our GitHub repository. + +* [**Connect to external Elastic resources**](connect-to-external-elastic-resources.md): Use custom `secrets` for the `elasticsearchRef` and `kibanaRef` parameters. + +ECK also facilitates configuration and operation activities with advanced features, such as: + +* [**Elastic Stack configuration policies**](elastic-stack-configuration-policies.md): Organize your {{es}} and {{kib}} configuration settings through `StackConfigPolicy` resources that can be referenced within your deployments. This helps to keep your manifests simplified. + +::::{important} +Explore the [Security](/deploy-manage/security.md) and [Users and roles](/deploy-manage/users-roles.md) sections to to learn more about how to secure and control access your deployments. +:::: diff --git a/deploy-manage/deploy/cloud-on-k8s/configure-eck.md b/deploy-manage/deploy/cloud-on-k8s/configure-eck.md index 9497496909..c2c6cc018d 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configure-eck.md +++ b/deploy-manage/deploy/cloud-on-k8s/configure-eck.md @@ -1,93 +1,96 @@ --- +navigation_title: Apply configuration settings +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-operator-config.html --- -# Configure ECK [k8s-operator-config] - -ECK can be configured using either command line flags or environment variables. - -| Flag | Default | Description | -| --- | --- | --- | -| `ca-cert-rotate-before` | `24h` | Duration representing how long before expiration CA certificates should be re-issued. | -| `ca-cert-validity` | `8760h` | Duration representing the validity period of a generated CA certificate. | -| `ca-dir` | `""` | Path to a directory containing a CA certificate (tls.crt) and its associated private key (tls.key) to be used for all managed resources. Effectively disables the CA rotation and validity options. | -| `cert-rotate-before` | `24h` | Duration representing how long before expiration TLS certificates should be re-issued. | -| `cert-validity` | `8760h` | Duration representing the validity period of a generated TLS certificate. | -| `config` | `""` | Path to a file containing the operator configuration. | -| `container-registry` | `docker.elastic.co` | Container registry to use for pulling Elastic Stack container images. | -| `container-repository` | `""` | Container repository to use for pulling Elastic Stack container images. | -| `container-suffix` | `""` | Suffix to be appended to container images by default. Cannot be combined with `--ubi-only` flag. | -| `disable-config-watch` | `false` | Watch the configuration file for changes and restart to apply them. Only effective when the `--config` flag is used to set the configuration file. | -| `disable-telemetry` | `false` | Disable periodically updating ECK telemetry data for Kibana to consume. | -| `elasticsearch-client-timeout` | `180s` | Default timeout for requests made by the Elasticsearch client. | -| `enable-leader-election` | `true` | Enable leader election. Must be set to true if using multiple replicas of the operator | -| `enable-tracing` | `false` | Enable APM tracing in the operator process. Use environment variables to configure APM server URL, credentials, and so on. Check [Apm Go Agent reference](https://www.elastic.co/guide/en/apm/agent/go/current/configuration.html) for details. | -| `enable-webhook` | `false` | Enables a validating webhook server in the operator process. | -| `enforce-rbac-on-refs` | `false` | Enables restrictions on cross-namespace resource association through RBAC. | -| `exposed-node-labels` | `""` | List of Kubernetes node labels which are allowed to be copied as annotations on the Elasticsearch Pods. Check [Topology spread constraints and availability zone awareness](advanced-elasticsearch-node-scheduling.md#k8s-availability-zone-awareness) for more details. | -| `ip-family` | `""` | Set the IP family to use. Possible values: IPv4, IPv6, "" (= auto-detect) | -| `kube-client-qps` | `0` | Set the maximum number of queries per second to the Kubernetes API. Default value is inherited from the [Go client](https://github.com/kubernetes/client-go/blob/e6538dd42b4fe55b6c754e41c66b43133ba41a59/rest/config.go#L44). | -| `kube-client-timeout` | `60s` | Set the request timeout for Kubernetes API calls made by the operator. | -| `log-verbosity` | `0` | Verbosity level of logs. `-2`=Error, `-1`=Warn, `0`=Info, `0` and above=Debug. | -| `manage-webhook-certs` | `true` | Enables automatic webhook certificate management. | -| `max-concurrent-reconciles` | `3` | Maximum number of concurrent reconciles per controller (Elasticsearch, Kibana, APM Server). Affects the ability of the operator to process changes concurrently. | -| `metrics-cert-dir` | `"{{TempDir}}/k8s-metrics-server/serving-certs"` | Location of TLS certs for the metrics server. Directory needs to contain tls.key and tls.crt. If empty self-signed certificates are used. Only effective when combined with metrics-port and metrics-secure. | -| `metrics-host` | `0.0.0.0` | The host to which the operator should bind to serve metrics in the Prometheus format. Will be combined with metrics-port. | -| `metrics-port` | `0` | Prometheus metrics port. Set to 0 to disable the metrics endpoint. | -| `metrics-secure` | `false` | Enables TLS for the metrics server. Only effective combined with metrics-port. | -| `namespaces` | `""` | Namespaces in which this operator should manage resources. Accepts multiple comma-separated values. Defaults to all namespaces if empty or unspecified. | -| `operator-namespace` | `""` | Namespace the operator runs in. Required. | -| `password-hash-cache-size` | `5 x max-concurrent-reconciles` | Sets the size of the password hash cache. Caching is disabled if explicitly set to 0 or any negative value. | -| `set-default-security-context` | `auto-detect` | Enables adding a default Pod Security Context to Elasticsearch Pods in Elasticsearch `8.0.0` and later. `fsGroup` is set to `1000` by default to match Elasticsearch container default UID. This behavior might not be appropriate for OpenShift and PSP-secured Kubernetes clusters, so it can be disabled. | -| `ubi-only` | `false` | Use only UBI container images to deploy Elastic Stack applications. UBI images are only available from 7.10.0 onward. Cannot be combined with `--container-suffix` flag. | -| `validate-storage-class` | `true` | Specifies whether the operator should retrieve storage classes to verify volume expansion support. Can be disabled if cluster-wide storage class RBAC access is not available. | -| `webhook-cert-dir` | `"{{TempDir}}/k8s-webhook-server/serving-certs"` | Path to the directory that contains the webhook server key and certificate. | -| `webhook-name` | `"elastic-webhook.k8s.elastic.co"` | Name of the Kubernetes ValidatingWebhookConfiguration resource. Only used when `enable-webhook` is true. | -| `webhook-secret` | `""` | K8s secret mounted into the path designated by webhook-cert-dir to be used for webhook certificates. | -| `webhook-port` | `9443` | Port to listen for incoming validation requests. | - -Unless noted otherwise, environment variables can be used instead of flags to configure the operator as well. Simply convert the flag name to upper case and replace any dashes (`-`) with underscores (`_`). For example, the `log-verbosity` flag can be set by an environment variable named `LOG_VERBOSITY`. - -Duration values should be specified as numeric values suffixed by the time unit. For example, a duration of 10 hours should be specified as `10h`. Acceptable time unit suffixes are: - -| Suffix | Unit | -| --- | --- | -| `ms` | Milliseconds | -| `s` | Seconds | -| `m` | Minutes | -| `h` | Hours | +# Apply ECK configuration settings [k8s-operator-config] -If you have a large number of configuration options to specify, use the `--config` flag to point to a file containing those options. For example, assume you have a file named `eck-config.yaml` with the following content: +This page explains the various methods for configuring and applying ECK settings. + +::::{tip} +For a detailed list and description of all available settings in ECK, refer to asciidocalypse://reference/cloud/cloud-on-k8s/eck-configuration-flags.md. +:::: + +By default, the ECK installation includes a [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) with an `eck.yaml` key where you can add, remove, or update configuration settings. This ConfigMap is mounted into the operator’s container as a file, and provided to the application through the `--config` flag. + +::::{note} +If you use [Operator Lifecycle Manager](https://github.com/operator-framework/operator-lifecycle-manager), refer to [Configure ECK under Operator Lifecycle Manager](#k8s-operator-config-olm) +:::: + +To configure ECK settings, follow the instructions in the next sections depending on whether you installed ECK through the Helm chart or the YAML manifests. + +## Using the operator Helm chart + +If you installed ECK through the Helm chart commands listed in [](./install-using-helm-chart.md), add your configuration parameters under the `config` key in your values file, or set them inline using the equivalent `--set config.=` flags when updating or installing the release. + +For example, to add the `ca-cert-validity` setting with a value of `43800h`, you can use any of the following methods: + +### Option 1: Use a values file and reference it in the helm upgrade command: + +Create a values file with the following content: ```yaml -log-verbosity: 2 -metrics-port: 6060 -namespaces: [ns1, ns2, ns3] +config: + ca-cert-validity: 43800h ``` -The operator can be started using any of the following methods to achieve the same end result: +Then, update the installed release pointing to the values file: ```sh -./elastic-operator manager --config=eck-config.yaml +helm upgrade elastic-operator elastic/eck-operator -f my-values-file.yaml -n elastic-system ``` -```sh -./elastic-operator manager --log-verbosity=2 --metrics-port=6060 --namespaces=ns1,ns2,ns3 -``` +### Option 2: Use `--set` in the helm upgrade command ```sh -LOG_VERBOSITY=2 METRICS_PORT=6060 NAMESPACES="ns1,ns2,ns3" ./elastic-operator manager +helm upgrade elastic-operator elastic/eck-operator --set config.ca-cert-validity=43800h -n elastic-system ``` -If you use a combination of all or some of the these methods, the descending order of precedence in case of a conflict is as follows: +## Using the operator YAML manifests -* Flag -* Environment variable -* File +If you installed ECK using the manifests and the commands listed in [Deploy ECK](./install-using-yaml-manifest-quickstart.md), you can configure it by editing the `eck.yaml` key of the `elastic-operator` ConfigMap. Add, remove or update any configuration setting there and the operator will restart automatically to apply the new changes unless the `--disable-config-watch` flag is set. + +You can update the ConfigMap directly using the command `kubectl edit configmap elastic-operator -n elastic-operator` or modify the installation manifests and reapply them with `kubectl apply -f `. -You can edit the `elastic-operator` ConfigMap to change the operator configuration. Unless the `--disable-config-watch` flag is set, the operator should restart automatically to apply the new changes. Alternatively, you can edit the `elastic-operator` StatefulSet and add flags to the `args` section — which will trigger an automatic restart of the operator pod by the StatefulSet controller. +The following shows the default `elastic-operator` ConfigMap, for reference purposes. Refer to asciidocalypse://reference/cloud/cloud-on-k8s/eck-configuration-flags.md for a complete list of available settings. +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: elastic-operator + namespace: elastic-system +data: + eck.yaml: |- + log-verbosity: 0 + metrics-port: 0 + container-registry: docker.elastic.co + max-concurrent-reconciles: 3 + ca-cert-validity: 8760h + ca-cert-rotate-before: 24h + cert-validity: 8760h + cert-rotate-before: 24h + disable-config-watch: false + exposed-node-labels: [topology.kubernetes.io/.*,failure-domain.beta.kubernetes.io/.*] + set-default-security-context: auto-detect + kube-client-timeout: 60s + elasticsearch-client-timeout: 180s + disable-telemetry: false + distribution-channel: all-in-one + validate-storage-class: true + enable-webhook: true + webhook-name: elastic-webhook.k8s.elastic.co + webhook-port: 9443 + operator-namespace: elastic-system + enable-leader-election: true + elasticsearch-observation-interval: 10s + ubi-only: false +``` + +Alternatively, you can edit the `elastic-operator` StatefulSet and add flags to the `args` section of the operator container — which will trigger an automatic restart of the operator pod by the StatefulSet controller. ## Configure ECK under Operator Lifecycle Manager [k8s-operator-config-olm] @@ -127,7 +130,7 @@ If you use [Operator Lifecycle Manager (OLM)](https://github.com/operator-framew name: elastic-cloud-eck source: elastic-operators sourceNamespace: openshift-marketplace - startingCSV: elastic-cloud-eck.v2.16.1 + startingCSV: elastic-cloud-eck.v{{eck_version}} config: volumes: - name: config @@ -139,4 +142,42 @@ If you use [Operator Lifecycle Manager (OLM)](https://github.com/operator-framew readOnly: true ``` +## Advanced configuration methods + +ECK can be configured using either command-line flags, environment variables or a file containing the operator configuration, pointed by `--config` flag. + +::::{important} +For most use cases, Elastic recommends configuring ECK through the `elastic-operator` ConfigMap, which is included by default in all installation methods. +This section provides a low-level overview of alternative configuration methods, primarily intended for developers or advanced users who might need to start the operator binary manually or adjust its configuration without modifying the ConfigMap. The implementation of these methods through Kubernetes manifests is out of the scope of this document. +:::: + +To pass configuration options as environment variables, convert the flag name to upper case and replace any dashes (`-`) with underscores (`_`). For example, the `log-verbosity` flag can be set by an environment variable named `LOG_VERBOSITY`. + +If you use a combination of all or some of the these methods, the descending order of precedence in case of a conflict is as follows: + +* Flag +* Environment variable +* File + +If you have a large number of configuration options to specify, use the `--config` flag to point to a file containing those options. For example, assume you have a file named `eck-config.yaml` with the following content: + +```yaml +log-verbosity: 2 +metrics-port: 6060 +namespaces: [ns1, ns2, ns3] +``` + +The operator can be started using any of the following methods to achieve the same end result: + +```sh +./elastic-operator manager --config=eck-config.yaml +``` + +```sh +./elastic-operator manager --log-verbosity=2 --metrics-port=6060 --namespaces=ns1,ns2,ns3 +``` + +```sh +LOG_VERBOSITY=2 METRICS_PORT=6060 NAMESPACES="ns1,ns2,ns3" ./elastic-operator manager +``` diff --git a/deploy-manage/deploy/cloud-on-k8s/configure-validating-webhook.md b/deploy-manage/deploy/cloud-on-k8s/configure-validating-webhook.md index 25f5523f56..fb35b29fe9 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configure-validating-webhook.md +++ b/deploy-manage/deploy/cloud-on-k8s/configure-validating-webhook.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-webhook.html --- @@ -24,7 +26,7 @@ Validating webhooks are defined using a `ValidatingWebhookConfiguration` object When using the default `operator.yaml` manifest, ECK is installed with a `ValidatingWebhookConfiguration` configured as follows: -* Validate all known Elastic custom resources (Elasticsearch, Kibana, APM Server, Enterprise Search, Beats, Elastic Agent, Elastic Maps Server, and Logstash) on create and update. +* Validate all known Elastic custom resources (Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash) on create and update. * The operator itself is the webhook server — which is exposed through a service named `elastic-webhook-server` in the `elastic-system` namespace. * The operator generates a certificate for the webhook and stores it in a secret named `elastic-webhook-server-cert` in the `elastic-system` namespace. This certificate is automatically rotated by the operator when it is due to expire. diff --git a/deploy-manage/deploy/cloud-on-k8s/configure.md b/deploy-manage/deploy/cloud-on-k8s/configure.md index dd31d9655e..d7c8f7f220 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configure.md +++ b/deploy-manage/deploy/cloud-on-k8s/configure.md @@ -1,18 +1,43 @@ --- +navigation_title: Configure +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-operating-eck.html --- -# Configure [k8s-operating-eck] +# Configure ECK [k8s-operating-eck] -* [*Configure ECK*](configure-eck.md) -* [*Required RBAC permissions*](required-rbac-permissions.md) -* [*Configure the validating webhook*](configure-validating-webhook.md) -* [*Configure the metrics endpoint*](../../monitor/orchestrators/eck-metrics-configuration.md) -* [*Restrict cross-namespace resource associations*](restrict-cross-namespace-resource-associations.md) -* [*Manage licenses in ECK*](../../license/manage-your-license-in-eck.md) -* [*Install ECK*](install.md) -* [*Upgrade ECK*](../../upgrade/orchestrator/upgrade-cloud-on-k8s.md) -* [*Uninstall ECK*](../../uninstall/uninstall-elastic-cloud-on-kubernetes.md) -* [*Running in air-gapped environments*](air-gapped-install.md) +This section covers ECK configuration mechanisms and use cases, starting with the basic setup of the operator using the provided `ConfigMap` and extending to more advanced configuration scenarios that require detailed procedures. +::::{tip} +This content focuses on ECK operator configuration. For details on available features and how to configure your {{es}} and {{kib}} deployments, refer to [](./configure-deployments.md). +:::: + +The following guides cover common ECK configuration tasks: + +* [](./configure-eck.md): Apply configuration changes, such the CA certificate validity period, the namespaces where the operator is allowed to work, or the log verbosity level for ECK. + +* [Configure the validating webhook](configure-validating-webhook.md): Enable or disable the webhook, and configure multiple SSL certificate generation options. + +* [Restrict cross-namespace resource associations](restrict-cross-namespace-resource-associations.md): Configure access control rules for cross-namespace associations. This functionality is disabled by default. + +* [Create custom images](./create-custom-images.md): Use your own images with {{es}} plugins already installed rather than installing them through init containers. + +* [Service meshes](./service-meshes.md): Connect ECK and managed Elastic Stack applications to some of the most popular [service mesh](https://www.cncf.io/blog/2017/04/26/service-mesh-critical-component-cloud-native-stack/) implementations in the Kubernetes ecosystem. + +* [Network policies](./network-policies.md): Use [Kubernetes network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) to isolate pods by restricting incoming and outgoing network connections to a trusted set of sources and destinations. + +* [](./webhook-namespace-selectors.md): Restrict the namespaces that the validation webhook applies to, allowing multiple operators to coexist efficiently in the same cluster. + +Other sections of the Elastic documentation cover additional topics related to ECK configuration: + +* **Monitoring** + * [Configure the metrics endpoint](/deploy-manage/monitor/orchestrators/eck-metrics-configuration.md) (monitor an orchestrator) + +* **Licensing** + * [Manage licenses in ECK](../../license/manage-your-license-in-eck.md) + +* **Maintenance** + * [Upgrade ECK](../../upgrade/orchestrator/upgrade-cloud-on-k8s.md) + * [Uninstall ECK](../../uninstall/uninstall-elastic-cloud-on-kubernetes.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/connect-to-apm-server.md b/deploy-manage/deploy/cloud-on-k8s/connect-to-apm-server.md index 1eb744a0be..83bb204532 100644 --- a/deploy-manage/deploy/cloud-on-k8s/connect-to-apm-server.md +++ b/deploy-manage/deploy/cloud-on-k8s/connect-to-apm-server.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-connecting.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/connect-to-external-elastic-resources.md b/deploy-manage/deploy/cloud-on-k8s/connect-to-external-elastic-resources.md index 4353ef8e6b..9ec084e1fc 100644 --- a/deploy-manage/deploy/cloud-on-k8s/connect-to-external-elastic-resources.md +++ b/deploy-manage/deploy/cloud-on-k8s/connect-to-external-elastic-resources.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-connect-to-unmanaged-resources.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/create-custom-images.md b/deploy-manage/deploy/cloud-on-k8s/create-custom-images.md index d789ef3ef2..8f20ed39e4 100644 --- a/deploy-manage/deploy/cloud-on-k8s/create-custom-images.md +++ b/deploy-manage/deploy/cloud-on-k8s/create-custom-images.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-custom-images.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/custom-configuration-files-plugins.md b/deploy-manage/deploy/cloud-on-k8s/custom-configuration-files-plugins.md index 20d08630cb..1b966ab92c 100644 --- a/deploy-manage/deploy/cloud-on-k8s/custom-configuration-files-plugins.md +++ b/deploy-manage/deploy/cloud-on-k8s/custom-configuration-files-plugins.md @@ -1,11 +1,13 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-bundles-plugins.html --- # Custom configuration files and plugins [k8s-bundles-plugins] -To run Elasticsearch with specific plugins or configuration files installed on ECK, you have two options. Each option has its own pros and cons. +To run Elasticsearch with specific plugins or configuration files installed on ECK, you have multiple options. Each option has its own pros and cons. 1. Create a custom container image with the required plugins and configuration files. @@ -33,9 +35,26 @@ To run Elasticsearch with specific plugins or configuration files installed on E * Each Elasticsearch node needs to repeat the download, wasting bandwidth and slowing startup. * Deployment manifests are more complicated. +3. Use ConfigMaps or Secrets together with volumes and volume mounts for configuration files. + + * **Pros** + + * Best choice for injecting configuration files into your {{es}} nodes. + * Follows standard Kubernetes methodology to mount files into Pods. + + * **Cons** + + * Not valid for plugins installation. + * Requires to maintain the ConfigMaps or Secrets with the content of the files. + +The following sections provide examples for each of the mentioned options. + +## Create a custom image Refer to [Creating custom images](create-custom-images.md) for instructions on how to build custom Docker images based on the official Elastic images. +## Use init containers for plugins installation + The following example describes option 2, using a repository plugin. To install the plugin before the Elasticsearch nodes start, use an init container to run the [plugin installation tool](https://www.elastic.co/guide/en/elasticsearch/plugins/current/installation.html). ```yaml @@ -55,9 +74,9 @@ spec: bin/elasticsearch-plugin install --batch repository-azure ``` -To install custom configuration files you can use volumes and volume mounts. +### Note when using Istio [istio-note] -The next example shows how to add a synonyms file for the [synonym token filter](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-synonym-tokenfilter.html) in Elasticsearch. But you can use the same approach for any kind of file you want to mount into the configuration directory of Elasticsearch. +When using Istio, init containers do **not** have network access, as the Envoy sidecar that provides network connectivity is not started yet. In this scenario, custom containers are the best option. If custom containers are simply not a viable option, then it is possible to adjust the startup command for the {{es}} container itself to run the plugin installation before starting {{es}}, as the following example describes. Note that this approach will require updating the startup command if it changes in the {{es}} image, which could potentially cause failures during upgrades. ```yaml spec: @@ -67,24 +86,45 @@ spec: podTemplate: spec: containers: - - name: elasticsearch <1> - volumeMounts: - - name: synonyms - mountPath: /usr/share/elasticsearch/config/dictionaries - volumes: - - name: synonyms - configMap: - name: synonyms <2> + - name: elasticsearch + command: + - /usr/bin/env + - bash + - -c + - | + #!/usr/bin/env bash + set -e + bin/elasticsearch-plugin remove --purge repository-s3 || true + bin/elasticsearch-plugin install --batch repository-s3 + /bin/tini -- /usr/local/bin/docker-entrypoint.sh ``` -1. Elasticsearch runs by convention in a container called *elasticsearch*. -2. Assuming you have created a config map in the same namespace as Elasticsearch with the name *synonyms* containing the synonyms file(s). +## Use a volume and volume mount together with a ConfigMap or Secret + +To install custom configuration files you can: +1. Add the configuration data into a ConfigMap or Secret. +2. Use volumes and volume mounts in your manifest to mount the contents of the ConfigMap or Secret as files in your {{es}} nodes. -$$$istio-note$$$ -**Note when using Istio** +The next example shows how to add a synonyms file for the [synonym token filter](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-synonym-tokenfilter.html) in Elasticsearch. But you can **use the same approach for any kind of file you want to mount into the configuration directory of Elasticsearch**, like adding CA certificates of external systems. -When using Istio, init containers do **not** have network access, as the Envoy sidecar that provides network connectivity is not started yet. In this scenario, custom containers are the best option. If custom containers are simply not a viable option, then it is possible to adjust the startup command for the elasticsearch container itself to run the plugin installation before starting Elasticsearch, as the following example describes. Note that this approach will require updating the startup command if it changes in the Elasticsearch image, which could potentially cause failures during upgrades. +1. Create the ConfigMap or Secret with the data: + +There are multiple ways to create and mount [ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/) and [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) on Kubernetes. Refer to the official documentation for more details. + +This example shows how to create a ConfigMap named `synonyms` with the content of a local file named `my-synonyms.txt` added into the `synonyms-elasticsearch.txt` key of the ConfigMap. + +```sh +kubectl create configmap synonyms -n --from-file=my-synonyms.txt=synonyms-elasticsearch.txt +``` + +::::{tip} +Create the ConfigMap or Secret in the same namespace where your {{es}} cluster runs. +:::: + +2. Declare the ConfigMap as a volume and mount it in the Elasticsearch containers. + +In this example, modify your {{es}} manifest to mount the contents of the `synonyms` ConfigMap into `/usr/share/elasticsearch/config/dictionaries` on the {{es}} nodes. ```yaml spec: @@ -94,15 +134,19 @@ spec: podTemplate: spec: containers: - - name: elasticsearch - command: - - /usr/bin/env - - bash - - -c - - | - #!/usr/bin/env bash - set -e - bin/elasticsearch-plugin remove --purge repository-s3 || true - bin/elasticsearch-plugin install --batch repository-s3 - /bin/tini -- /usr/local/bin/docker-entrypoint.sh + - name: elasticsearch <1> + volumeMounts: + - name: synonyms + mountPath: /usr/share/elasticsearch/config/dictionaries <2> + volumes: + - name: synonyms + configMap: <3> + name: synonyms <4> ``` + +1. Elasticsearch runs by convention in a container called `elasticsearch`. Do not change that value. +2. Use always a path under `/usr/share/elasticsearch/config`. +3. Use `secret` instead of `configMap` if you used a secret to store the data. +4. The ConfigMap name must be the same as the ConfigMap created in the previous step. + +After the changes are applied, {{es}} nodes should be able to access `dictionaries/synonyms-elasticsearch.txt` and use it in any [configuration setting](./node-configuration.md). diff --git a/deploy-manage/deploy/cloud-on-k8s/customize-pods.md b/deploy-manage/deploy/cloud-on-k8s/customize-pods.md index c48aac2c3b..3bee229312 100644 --- a/deploy-manage/deploy/cloud-on-k8s/customize-pods.md +++ b/deploy-manage/deploy/cloud-on-k8s/customize-pods.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-customize-pods.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/deploy-an-orchestrator.md b/deploy-manage/deploy/cloud-on-k8s/deploy-an-orchestrator.md index d942cbfa4e..b64f751985 100644 --- a/deploy-manage/deploy/cloud-on-k8s/deploy-an-orchestrator.md +++ b/deploy-manage/deploy/cloud-on-k8s/deploy-an-orchestrator.md @@ -1,18 +1,22 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html --- # Deploy an orchestrator [k8s-quickstart] -With Elastic Cloud on Kubernetes (ECK) you can extend the basic Kubernetes orchestration capabilities to easily deploy, secure, upgrade your {{es}} cluster, and much more. +With Elastic Cloud on Kubernetes (ECK), you can extend the basic Kubernetes orchestration capabilities to easily deploy, secure, upgrade your {{es}} cluster, along with other Elastic applications. -Eager to get started? This quickstart guide shows you how to: +In this section, you'll learn how to do the following: -* [Deploy ECK in your Kubernetes cluster](install-using-yaml-manifest-quickstart.md) -* [Deploy an {{es}} cluster](elasticsearch-deployment-quickstart.md) -* [Deploy a {{kib}} instance](kibana-instance-quickstart.md) -* [Update your deployment](update-deployments.md) +- [**Installing the ECK Operator**](./install.md): Learn different installation methods, including Helm and YAML manifests. +- [**Deploying in air-gapped environments**](./air-gapped-install.md): Follow best practices for installing and operating ECK in restricted networks. +- [**Configuring ECK**](./configure.md): Understand the available configuration options to optimize your ECK deployment. -Afterwards, you can find further sample resources [in the project repository](https://github.com/elastic/cloud-on-k8s/tree/2.16/config/samples) or by checking out [our recipes](recipes.md). +To learn how to deploy {{es}}, {{kib}}, or other Elastic applications using ECK, refer to [](./manage-deployments.md). +::::{tip} +For a quickstart experience covering installation of ECK and deployment of an {{es}} cluster with a {{kib}} instance, refer to [](../cloud-on-k8s.md#eck-quickstart) +:::: \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md b/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md index b3f6bb82fe..077fd70698 100644 --- a/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md +++ b/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md @@ -1,6 +1,12 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-autopilot.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-autopilot-setting-virtual-memory.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-autopilot-deploy-the-operator.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-autopilot-deploy-elasticsearch.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-autopilot-deploy-agent-beats.html --- # Deploy ECK on GKE Autopilot [k8s-autopilot] @@ -8,17 +14,54 @@ mapped_pages: This page shows how to run ECK on GKE Autopilot. 1. It is recommended that each Kubernetes host’s virtual memory kernel settings be modified. Refer to [Virtual memory](virtual-memory.md). -2. It is recommended that Elasticsearch Pods have an `initContainer` that waits for virtual memory settings to be in place. Refer to [Deploy an Elasticsearch instance](k8s-autopilot-deploy-elasticsearch.md). -3. For Elastic Agent/Beats there are storage limitations to be considered. Refer to [Deploy a standalone Elastic Agent and/or Beats](k8s-autopilot-deploy-agent-beats.md) -4. Ensure you are using a node class that is applicable for your workload by adding a `cloud.google.com/compute-class` label in a `nodeSelector`. Refer to [GKE Autopilot documentation.](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-compute-classes) +2. It is recommended that Elasticsearch Pods have an `initContainer` that waits for virtual memory settings to be in place. +3. For Elastic Agent/Beats there are storage limitations to be considered. +4. Ensure you are using a node class that is applicable for your workload by adding a `cloud.google.com/compute-class` label in a `nodeSelector`. Refer to [GKE Autopilot documentation.](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-compute-classes). - * [Ensuring virtual memory kernel settings](k8s-autopilot-setting-virtual-memory.md) - * [Installing the ECK Operator](k8s-autopilot-deploy-operator.md) - * [Deploy an Elasticsearch instance](k8s-autopilot-deploy-elasticsearch.md) - * [Deploy a standalone Elastic Agent and/or Beats](k8s-autopilot-deploy-agent-beats.md) +## Ensuring virtual memory kernel settings [k8s-autopilot-setting-virtual-memory] +If you are intending to run production workloads on GKE Autopilot then `vm.max_map_count` should be set. The recommended way to set this kernel setting on the Autopilot hosts is with a `Daemonset` as described in the [Virtual memory](virtual-memory.md) section. You must be running at least version 1.25 when on the `regular` channel or using the `rapid` channel, which currently runs version 1.27. +::::{warning} +Only use the provided `Daemonset` exactly as specified or it could be rejected by the Autopilot control plane. +:::: +## Install the ECK Operator [k8s-autopilot-deploy-the-operator] +Refer to [*Install ECK*](install.md) for more information on installation options. +## Deploy an Elasticsearch cluster [k8s-autopilot-deploy-elasticsearch] + +Create an Elasticsearch cluster. If you are using the `Daemonset` described in the [Virtual memory](virtual-memory.md) section to set `max_map_count` you can add the `initContainer` below is also used to ensure the setting is set prior to starting Elasticsearch. + +```shell +cat < | -| Helm Charts | `app.kubernetes.io/name: elastic-operator`
| - -::::{note} -The examples in this section assume that the ECK operator has been installed using the Helm chart. -:::: - - - -## Kubernetes API server IP [k8s_kubernetes_api_server_ip] - -Run `kubectl get endpoints kubernetes -n default` to obtain the API server IP address for your cluster. - -::::{note} -The following examples assume that the Kubernetes API server IP address is `10.0.0.1`. -:::: - - - -## Isolating the operator [k8s-network-policies-operator-isolation] - -The minimal set of permissions required are as follows: - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 443 of the Kubernetes API server.
* UDP port 53 for DNS lookup.
* TCP port 9200 of {{es}} nodes on managed namespace.
| -| Ingress (incoming) | * TCP port 9443 for webhook requests from the Kubernetes API server.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: elastic-operator - namespace: elastic-system -spec: - egress: - - ports: - - port: 53 - protocol: UDP - - ports: - - port: 443 - protocol: TCP - to: - - ipBlock: - cidr: 10.0.0.1/32 - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchExpressions: - - key: eck.k8s.elastic.co/tenant - operator: In - values: - - team-a - - team-b - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - ingress: - - from: - - ipBlock: - cidr: 10.0.0.1/32 - ports: - - port: 9443 - protocol: TCP - podSelector: - matchLabels: - app.kubernetes.io/name: elastic-operator -``` - - -## Isolating {{es}} [k8s-network-policies-elasticsearch-isolation] - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9300 to other {{es}} nodes in the namespace (transport port).
* UDP port 53 for DNS lookup.
| -| Ingress (incoming) | * TCP port 9200 from the operator and other pods in the namespace.
* TCP port 9300 from other {{es}} nodes in the namespace (transport port).
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-elasticsearch - namespace: team-a -spec: - egress: - - ports: - - port: 9300 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 53 - protocol: UDP - ingress: - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/operator-name: elastic-operator - podSelector: - matchLabels: - app.kubernetes.io/name: elastic-operator - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - # [Optional] Allow ingress controller pods from the ingress-nginx namespace. - #- namespaceSelector: - # matchLabels: - # name: ingress-nginx - ports: - - port: 9200 - protocol: TCP - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - ports: - - port: 9300 - protocol: TCP - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch -``` - - -## Isolating {{kib}} [k8s-network-policies-kibana-isolation] - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* UDP port 53 for DNS lookup.
| -| Ingress (incoming) | * TCP port 5601 from other pods in the namespace.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-kibana - namespace: team-a -spec: - egress: - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - # [Optional] Restrict to a single {{es}} cluster named hulk. - # elasticsearch.k8s.elastic.co/cluster-name=hulk - - ports: - - port: 53 - protocol: UDP - # [Optional] If Agent is deployed, this is to allow Kibana to access the Elastic Package Registry (https://epr.elastic.co). - # - port: 443 - # protocol: TCP - ingress: - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - # [Optional] Allow ingress controller pods from the ingress-nginx namespace. - #- namespaceSelector: - # matchLabels: - # name: ingress-nginx - ports: - - port: 5601 - protocol: TCP - podSelector: - matchLabels: - common.k8s.elastic.co/type: kibana -``` - - -## Isolating APM Server [k8s-network-policies-apm-server-isolation] - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* UDP port 53 for DNS lookup.
| -| Ingress (incoming) | * TCP port 8200 from other pods in the namespace.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-apm-server - namespace: team-a -spec: - egress: - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 5601 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: kibana - - ports: - - port: 53 - protocol: UDP - ingress: - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - # [Optional] Allow ingress controller pods from the ingress-nginx namespace. - #- namespaceSelector: - # matchLabels: - # name: ingress-nginx - ports: - - port: 8200 - protocol: TCP - podSelector: - matchLabels: - common.k8s.elastic.co/type: apm-server -``` - - -## Isolating Enterprise Search [k8s-network-policies-enterprise-search-isolation] - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* UDP port 53 for DNS lookup.
| -| Ingress (incoming) | * TCP port 3002 from other pods in the namespace.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-enterprise-search - namespace: team-a -spec: - egress: - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 53 - protocol: UDP - ingress: - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - # [Optional] Allow ingress controller pods from the ingress-nginx namespace. - #- namespaceSelector: - # matchLabels: - # name: ingress-nginx - ports: - - port: 3002 - protocol: TCP - podSelector: - matchLabels: - common.k8s.elastic.co/type: enterprise-search -``` - - -## Isolating {{beats}} [k8s-network-policies-beats-isolation] - -::::{note} -Some {{beats}} may require additional access rules than what is listed here. For example, {{heartbeat}} will require a rule to allow access to the endpoint it is monitoring. -:::: - - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* UDP port 53 for DNS lookup.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-beats - namespace: team-a -spec: - egress: - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 5601 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: kibana - - ports: - - port: 53 - protocol: UDP - podSelector: - matchLabels: - common.k8s.elastic.co/type: beat -``` - - -## Isolating {{agent}} and {{fleet}} [k8s-network-policies-agent-isolation] - -::::{note} -Some {{agent}} policies may require additional access rules other than those listed here. -:::: - - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* TCP port 8220 to {{fleet}} instances in the namespace.
* UDP port 53 for DNS lookup.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-agent - namespace: team-a -spec: - egress: - - ports: - - port: 8220 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: agent - - ports: - - port: 5601 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: kibana - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 53 - protocol: UDP - - ports: - - port: 443 - protocol: TCP - to: - - ipBlock: - cidr: 10.0.0.1/32 - ingress: - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - ports: - - port: 8220 - protocol: TCP - podSelector: - matchLabels: - common.k8s.elastic.co/type: agent -``` - - -## Isolating {{ls}} [k8s-network-policies-logstash-isolation] - -::::{note} -{{ls}} may require additional access rules than those listed here, depending on plugin usage. -:::: - - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* UDP port 53 for DNS lookup.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-logstash - namespace: team-a -spec: - egress: - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 53 - protocol: UDP - podSelector: - matchLabels: - common.k8s.elastic.co/type: logstash -``` - - diff --git a/deploy-manage/deploy/cloud-on-k8s/kibana-configuration.md b/deploy-manage/deploy/cloud-on-k8s/kibana-configuration.md index af59b323ae..086b015f6d 100644 --- a/deploy-manage/deploy/cloud-on-k8s/kibana-configuration.md +++ b/deploy-manage/deploy/cloud-on-k8s/kibana-configuration.md @@ -1,11 +1,13 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-kibana.html --- # Kibana configuration [k8s-kibana] -The [quickstart](kibana-instance-quickstart.md) is a good starting point to quickly setup a {{kib}} instance with ECK. The following sections describe how to customize a {{kib}} deployment to suit your requirements. +The following sections describe how to customize a {{kib}} deployment to suit your requirements. * [Connect to an {{es}} cluster](k8s-kibana-es.md) @@ -26,9 +28,6 @@ The [quickstart](kibana-instance-quickstart.md) is a good starting point to quic * [Disable TLS](k8s-kibana-http-configuration.md#k8s-kibana-http-disable-tls) * [Install {{kib}} plugins](k8s-kibana-plugins.md) - - - - +* [Autoscaling stateless applications](../../autoscaling/autoscaling-stateless-applications-on-eck.md): Use [Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) for {{kib}} or other stateless applications. diff --git a/deploy-manage/deploy/cloud-on-k8s/kibana-instance-quickstart.md b/deploy-manage/deploy/cloud-on-k8s/kibana-instance-quickstart.md index 680343c7ac..57a8238fb6 100644 --- a/deploy-manage/deploy/cloud-on-k8s/kibana-instance-quickstart.md +++ b/deploy-manage/deploy/cloud-on-k8s/kibana-instance-quickstart.md @@ -1,9 +1,12 @@ --- +navigation_title: Deploy a Kibana instance +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-deploy-kibana.html --- -# Kibana instance quickstart [k8s-deploy-kibana] +# Deploy a Kibana instance [k8s-deploy-kibana] To deploy a simple [{{kib}}](https://www.elastic.co/guide/en/kibana/current/introduction.html#introduction) specification, with one {{kib}} instance: @@ -53,7 +56,7 @@ To deploy a simple [{{kib}}](https://www.elastic.co/guide/en/kibana/current/intr kubectl port-forward service/quickstart-kb-http 5601 ``` - Open `https://localhost:5601` in your browser. Your browser will show a warning because the self-signed certificate configured by default is not verified by a known certificate authority and not trusted by your browser. You can temporarily acknowledge the warning for the purposes of this quick start but it is highly recommended that you [configure valid certificates](tls-certificates.md#k8s-setting-up-your-own-certificate) for any production deployments. + Open `https://localhost:5601` in your browser. Your browser will show a warning because the self-signed certificate configured by default is not verified by a known certificate authority and not trusted by your browser. You can temporarily acknowledge the warning for the purposes of this quick start but it is highly recommended that you [configure valid certificates](/deploy-manage/security/secure-http-communications.md#k8s-setting-up-your-own-certificate) for any production deployments. Login as the `elastic` user. The password can be obtained with the following command: @@ -68,5 +71,12 @@ For a full description of each `CustomResourceDefinition` (CRD), refer to the [* kubectl describe crd kibana ``` -This completes the quickstart of deploying an {{kib}} instance on top of [the ECK operator](install-using-yaml-manifest-quickstart.md) and [deployed {{es}} cluster](elasticsearch-deployment-quickstart.md). We recommend continuing to [updating your deployment](update-deployments.md). For more {{kib}} configuration options, refer to [Running {{kib}} on ECK](kibana-configuration.md). +## Next steps + +This completes the quickstart of deploying an {{kib}} instance on top of [the ECK operator](install-using-yaml-manifest-quickstart.md) and [deployed {{es}} cluster](elasticsearch-deployment-quickstart.md). + +We recommend continuing to: +* [Updating your deployment](update-deployments.md). +* For more {{kib}} configuration options, refer to [{{kib}} configuration on ECK](kibana-configuration.md) and [](./configure-deployments.md). + diff --git a/deploy-manage/deploy/cloud-on-k8s/known-limitations.md b/deploy-manage/deploy/cloud-on-k8s/known-limitations.md index 0185402de1..f4cc067a15 100644 --- a/deploy-manage/deploy/cloud-on-k8s/known-limitations.md +++ b/deploy-manage/deploy/cloud-on-k8s/known-limitations.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-fleet-known-limitations.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/logstash-plugins.md b/deploy-manage/deploy/cloud-on-k8s/logstash-plugins.md index a3736cb6d6..33b0b2005e 100644 --- a/deploy-manage/deploy/cloud-on-k8s/logstash-plugins.md +++ b/deploy-manage/deploy/cloud-on-k8s/logstash-plugins.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-plugins.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/logstash.md b/deploy-manage/deploy/cloud-on-k8s/logstash.md index 3a614827b1..74b078da43 100644 --- a/deploy-manage/deploy/cloud-on-k8s/logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/manage-compute-resources.md b/deploy-manage/deploy/cloud-on-k8s/manage-compute-resources.md index b818afa956..d0bdbe21ed 100644 --- a/deploy-manage/deploy/cloud-on-k8s/manage-compute-resources.md +++ b/deploy-manage/deploy/cloud-on-k8s/manage-compute-resources.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-managing-compute-resources.html --- @@ -102,7 +104,7 @@ A [known Kubernetes issue](https://github.com/kubernetes/kubernetes/issues/51135 -### Set compute resources for Kibana, Enterprise Search, Elastic Maps Server, APM Server and Logstash [k8s-compute-resources-kibana-and-apm] +### Set compute resources for Kibana, Elastic Maps Server, APM Server and Logstash [k8s-compute-resources-kibana-and-apm] ```yaml apiVersion: kibana.k8s.elastic.co/v1 @@ -285,7 +287,6 @@ If `resources` is not defined in the specification of an object, then the operat | Beat | `300Mi` | `300Mi` | | Elastic Agent | `400Mi` | `400Mi` | | Elastic Maps Server | `200Mi` | `200Mi` | -| Enterprise Search | `4Gi` | `4Gi` | | Logstash | `2Gi` | `2Gi` | If the Kubernetes cluster is configured with [LimitRanges](https://kubernetes.io/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/) that enforce a minimum memory constraint, they could interfere with the operator defaults and cause object creation to fail. diff --git a/deploy-manage/deploy/cloud-on-k8s/manage-deployments.md b/deploy-manage/deploy/cloud-on-k8s/manage-deployments.md index 19c32a12f6..6e3f51aaec 100644 --- a/deploy-manage/deploy/cloud-on-k8s/manage-deployments.md +++ b/deploy-manage/deploy/cloud-on-k8s/manage-deployments.md @@ -1,7 +1,31 @@ +--- +applies: + eck: all +--- # Manage deployments -% What needs to be done: Write from scratch +This section provides detailed guidance on deploying, configuring, and managing Elasticsearch and Kibana within ECK. A **deployment** refers to an {{es}} cluster, optionally with one or more {{kib}} instances connected to it. -% GitHub issue: https://github.com/elastic/docs-projects/issues/357 +::::{tip} +This content focuses on Elasticsearch and Kibana deployments. To orchestrate other Elastic Stack applications such as APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash, refer to the [Orchestrating other Elastic Stack applications](./orchestrate-other-elastic-applications.md). +:::: -% Scope notes: To be decided... \ No newline at end of file +## What You'll Learn + +In this section, you'll learn how to perform the following tasks in ECK: + +- [**Deploy an Elasticsearch cluster**](./elasticsearch-deployment-quickstart.md): Orchestrate an {{es}} cluster in Kubernetes. +- [**Deploy Kibana instances**](./kibana-instance-quickstart.md): Set up and connect Kibana to an existing Elasticsearch cluster. +- [**Manage deployments using Elastic Stack Helm chart**](./managing-deployments-using-helm-chart.md): Use Helm to deploy clusters and other stack applications. +- [**Apply updates to your deployments**](./update-deployments.md): Modify existing deployments, scale clusters, and update configurations, while ensuring minimal disruption. +- [**Configure access to your deployments**](./accessing-services.md): Use and adapt Kubernetes services to your needs. +- [**Advanced configuration**](./configure-deployments.md): Explore available settings for Elasticsearch and Kibana, including storage, networking, security, and scaling options. + +For a complete reference on configuration possibilities for {{es}} and {{kib}}, see: + +- [](./elasticsearch-configuration.md) +- [](./kibana-configuration.md) + +Other references for managing deployments: + +* [**Upgrade the Elastic Stack version**](../../upgrade/deployment-or-cluster.md): Upgrade orchestrated applications on ECK. diff --git a/deploy-manage/deploy/cloud-on-k8s/managing-deployments-using-helm-chart.md b/deploy-manage/deploy/cloud-on-k8s/managing-deployments-using-helm-chart.md index 2780ad8eac..883c9d7805 100644 --- a/deploy-manage/deploy/cloud-on-k8s/managing-deployments-using-helm-chart.md +++ b/deploy-manage/deploy/cloud-on-k8s/managing-deployments-using-helm-chart.md @@ -1,9 +1,12 @@ --- +navigation_title: Elastic Stack Helm chart +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-stack-helm-chart.html --- -# Managing deployments using a Helm chart [k8s-stack-helm-chart] +# Elastic Stack Helm chart [k8s-stack-helm-chart] Starting from ECK 2.4.0, a Helm chart is available for managing Elastic Stack resources using the ECK Operator. It is available from the Elastic Helm repository and can be added to your Helm repository list by running the following command: @@ -16,21 +19,26 @@ helm repo update The minimum supported version of Helm is 3.2.0. :::: +The Elastic Stack (`eck-stack`) Helm chart is built on top of individual charts such as `eck-elasticsearch` and `eck-kibana`. For more details on its structure and dependencies, refer to the [chart repository](https://github.com/elastic/cloud-on-k8s/tree/main/deploy/eck-stack/). +The chart enables you to deploy the core components ({{es}} and {{kib}}) together, along with other {{stack}} applications if needed, under the same chart release. The following sections guide you through the installation process for multiple use cases. Choose the command that best fits your setup. -## Installing Elasticsearch and Kibana using the eck-stack Helm Chart [k8s-install-elasticsearch-kibana-helm] +::::{tip} +All the provided examples deploy the applications in a namespace named `elastic-stack`. Consider adapting the commands to your use case. +:::: + +## Elasticsearch and Kibana [k8s-install-elasticsearch-kibana-helm] -Similar to the [quickstart](elasticsearch-deployment-quickstart.md), the following section describes how to setup an Elasticsearch cluster with a simple Kibana instance managed by ECK, and how to customize a deployment using the eck-stack Helm chart’s values. +Similar to the quickstart examples for {{es}} and {{kib}}, this section describes how to setup an {{es}} cluster with a simple {{kib}} instance managed by ECK, and how to customize a deployment using the eck-stack Helm chart’s values. ```sh # Install an eck-managed Elasticsearch and Kibana using the default values, which deploys the quickstart examples. helm install es-kb-quickstart elastic/eck-stack -n elastic-stack --create-namespace ``` +### Customize Elasticsearch and Kibana installation with example values [k8s-eck-stack-helm-customize] -### Customizing Kibana and Elasticsearch using the eck-stack Helm Chart’s example values [k8s-eck-stack-helm-customize] - -There are example Helm values files for installing and managing a more advanced Elasticsearch and/or Kibana [in the project repository](https://github.com/elastic/cloud-on-k8s/tree/2.16/deploy/eck-stack/examples). +You can find example Helm values files for deploying and managing more advanced Elasticsearch and Kibana setups [in the project repository](https://github.com/elastic/cloud-on-k8s/tree/2.16/deploy/eck-stack/examples). To use one or more of these example configurations, use the `--values` Helm option, as seen in the following section. @@ -41,8 +49,7 @@ helm install es-quickstart elastic/eck-stack -n elastic-stack --create-namespace --values https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/deploy/eck-stack/examples/kibana/http-configuration.yaml ``` - -## Installing Fleet Server with Elastic Agents along with Elasticsearch and Kibana using the eck-stack Helm Chart [k8s-install-fleet-agent-elasticsearch-kibana-helm] +## Fleet Server with Elastic Agents along with Elasticsearch and Kibana [k8s-install-fleet-agent-elasticsearch-kibana-helm] The following section builds upon the previous section, and allows installing Fleet Server, and Fleet-managed Elastic Agents along with Elasticsearch and Kibana. @@ -52,8 +59,7 @@ helm install eck-stack-with-fleet elastic/eck-stack \ --values https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/deploy/eck-stack/examples/agent/fleet-agents.yaml -n elastic-stack ``` - -## Installing Logstash along with Elasticsearch, Kibana and Beats using the eck-stack Helm Chart [k8s-install-logstash-elasticsearch-kibana-helm] +## Logstash along with Elasticsearch, Kibana and Beats [k8s-install-logstash-elasticsearch-kibana-helm] The following section builds upon the previous sections, and allows installing Logstash along with Elasticsearch, Kibana and Beats. @@ -63,8 +69,7 @@ helm install eck-stack-with-logstash elastic/eck-stack \ --values https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/deploy/eck-stack/examples/logstash/basic-eck.yaml -n elastic-stack ``` - -## Installing a standalone Elastic APM Server along with Elasticsearch and Kibana using the eck-stack Helm Chart [k8s-install-apm-server-elasticsearch-kibana-helm] +## Standalone Elastic APM Server along with Elasticsearch and Kibana [k8s-install-apm-server-elasticsearch-kibana-helm] The following section builds upon the previous sections, and allows installing a standalone Elastic APM Server along with Elasticsearch and Kibana. @@ -74,12 +79,12 @@ helm install eck-stack-with-apm-server elastic/eck-stack \ --values https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/deploy/eck-stack/examples/apm-server/basic.yaml -n elastic-stack ``` -### Installing individual components of the Elastic Stack using the Helm Charts [k8s-eck-stack-individual-components] +## Install individual components of the Elastic Stack [k8s-eck-stack-individual-components] You can install individual components in one of two ways using the provided Helm Charts. 1. Using Helm values -2. Using the individual Helm Charts directly +2. Using the individual Helm Charts directly (not the `eck-stack` helm chart) **Using Helm values to install only Elasticsearch** @@ -93,12 +98,90 @@ helm install es-quickstart elastic/eck-stack -n elastic-stack --create-namespace helm install es-quickstart elastic/eck-elasticsearch -n elastic-stack --create-namespace ``` +## Adding Ingress to the Elastic stack [k8s-eck-stack-ingress] + +Both {{es}} and {{kib}} support [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/), which can be enabled using the following options: -### Adding Ingress to the Elastic stack using the Helm Charts [k8s-eck-stack-ingress] +**If an individual chart is used (not eck-stack)** -Both Elasticsearch and Kibana support Ingress, which can be enabled using the following options: +The following command installs an {{es}} cluster using the `eck-elasticsearch` chart and configures an ingress resource: ```sh -helm install es-quickstart elastic/eck-elasticsearch -n elastic-stack --create-namespace --set=ingress.enabled=true --set=ingress.hosts[0].host=elasticsearch.example.com --set=ingress.hosts[0].path="/" +helm install es-quickstart elastic/eck-elasticsearch -n elastic-stack --create-namespace \ + --set=ingress.enabled=true --set=ingress.hosts[0].host=elasticsearch.example.com --set=ingress.hosts[0].path="/" ``` +**If eck-stack chart is used** + +The following command deploys the basic {{es}} and {{kib}} example with ingress resources for both components: + +```sh +helm install es-kb-quickstart elastic/eck-stack -n elastic-stack --create-namespace \ + --set=eck-elasticsearch.ingress.enabled=true --set=eck-elasticsearch.ingress.hosts[0].host=elasticsearch.example.com --set=eck-elasticsearch.ingress.hosts[0].path="/" \ + --set=eck-kibana.ingress.enabled=true --set=eck-kibana.ingress.hosts[0].host=kibana.example.com --set=eck-kibana.ingress.hosts[0].path="/" +``` + +For illustration purposes, the ingress objects created by the previous command will look similar to the following: + +```yaml +# Source: eck-stack/charts/eck-elasticsearch/templates/ingress.yaml +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: elasticsearch + labels: + helm.sh/chart: eck-elasticsearch-0.14.1 + app.kubernetes.io/name: eck-elasticsearch + app.kubernetes.io/instance: es-kb-quickstart + app.kubernetes.io/managed-by: Helm +spec: + rules: + - host: "elasticsearch.example.com" + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: elasticsearch-es-http + port: + number: 9200 +--- +# Source: eck-stack/charts/eck-kibana/templates/ingress.yaml +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: es-kb-quickstart-eck-kibana + labels: + helm.sh/chart: eck-kibana-0.14.1 + app.kubernetes.io/name: eck-kibana + app.kubernetes.io/instance: es-kb-quickstart + app.kubernetes.io/managed-by: Helm +spec: + rules: + - host: "kibana.example.com" + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: es-kb-quickstart-eck-kibana-kb-http + port: + number: 5601 +``` + +## View available configuration options [k8s-install-helm-show-values-stack] + +You can view all configurable values of the Elastic Stack helm chart of the individual charts by running the following: + +```sh +helm show values elastic/eck-stack +helm show values elastic/eck-elasticsearch +helm show values elastic/eck-kibana +helm show values elastic/eck-agent +helm show values elastic/eck-beats +helm show values elastic/eck-apm-server +helm show values elastic/eck-fleet-server +helm show values elastic/eck-logstash +``` diff --git a/deploy-manage/deploy/cloud-on-k8s/map-data.md b/deploy-manage/deploy/cloud-on-k8s/map-data.md index cbff490667..7e22059b35 100644 --- a/deploy-manage/deploy/cloud-on-k8s/map-data.md +++ b/deploy-manage/deploy/cloud-on-k8s/map-data.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-maps-data.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/network-policies.md b/deploy-manage/deploy/cloud-on-k8s/network-policies.md index 0f0bbfdb82..2c26e06330 100644 --- a/deploy-manage/deploy/cloud-on-k8s/network-policies.md +++ b/deploy-manage/deploy/cloud-on-k8s/network-policies.md @@ -1,17 +1,25 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-network-policies.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s_prerequisites.html --- +% this section could be moved entirely to security. to be considered. + # Network policies [k8s-network-policies] -[Network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) allow you to isolate pods by restricting incoming and outgoing network connections to a trusted set of sources and destinations. This section describes how to use network policies to isolate the ECK operator and the {{stack}} applications to a set of namespaces to implement a form of soft multi-tenancy. Soft multi-tenancy is a term used to describe a scenario where a group of trusted users (different teams within an organization, for example) share a single resource such as a Kubernetes cluster. Note that network policies alone are not sufficient for security. You should complement them with strict RBAC policies, resource quotas, node taints, and other available security mechanisms to ensure that tenants cannot access, modify, or disrupt resources belonging to each other. +Kubernetes [network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) allow you to isolate pods by restricting incoming and outgoing network connections to a trusted set of sources and destinations. + +This section describes how to use network policies to isolate the ECK operator and the {{stack}} applications to a set of namespaces to implement a form of soft multi-tenancy. Soft multi-tenancy is a term used to describe a scenario where a group of trusted users (different teams within an organization, for example) share a single resource such as a Kubernetes cluster. + +Note that network policies alone are not sufficient for security. You should complement them with strict RBAC policies, resource quotas, node taints, and other available security mechanisms to ensure that tenants cannot access, modify, or disrupt resources belonging to each other. ::::{note} There are several efforts to support multi-tenancy on Kubernetes, including the [official working group for multi-tenancy](https://github.com/kubernetes-sigs/multi-tenancy) and community extensions such as [loft](https://loft.sh) and [kiosk](https://github.com/kiosk-sh/kiosk), that can make configuration and management easier. You might need to employ network policies such the ones described in this section to have fine-grained control over {{stack}} applications deployed by your tenants. :::: - The following sections assume that the operator is installed in the `elastic-system` namespace with the [`namespaces` configuration](configure-eck.md) set to `team-a,team-b`. Each namespace is expected to be labelled as follows: ```sh @@ -20,4 +28,405 @@ kubectl label namespace team-a eck.k8s.elastic.co/tenant=team-a kubectl label namespace team-b eck.k8s.elastic.co/tenant=team-b ``` +## Prerequisites [k8s_prerequisites] + +To set up the network policies correctly you must know the operator Pod selector and the Kubernetes API server IP. They may vary depending on your environment and how the operator has been installed. + +### Operator Pod selector [k8s_operator_pod_selector] + +The operator Pod label depends on how the operator has been installed. Check the following table to know which label name is used in the network policies. + +| Installation method | Pod selector | +| --- | --- | +| YAML manifests | `control-plane: elastic-operator`
| +| Helm Charts | `app.kubernetes.io/name: elastic-operator`
| + +::::{note} +The examples in this section assume that the ECK operator has been installed using the Helm chart. +:::: + +### Kubernetes API server IP [k8s_kubernetes_api_server_ip] + +Run `kubectl get endpoints kubernetes -n default` to obtain the API server IP address for your cluster. + +::::{note} +The following examples assume that the Kubernetes API server IP address is `10.0.0.1`. +:::: + +## Isolating the operator [k8s-network-policies-operator-isolation] + +The minimal set of permissions required are as follows: + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 443 of the Kubernetes API server.
* UDP port 53 for DNS lookup.
* TCP port 9200 of {{es}} nodes on managed namespace.
| +| Ingress (incoming) | * TCP port 9443 for webhook requests from the Kubernetes API server.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: elastic-operator + namespace: elastic-system +spec: + egress: + - ports: + - port: 53 + protocol: UDP + - ports: + - port: 443 + protocol: TCP + to: + - ipBlock: + cidr: 10.0.0.1/32 + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchExpressions: + - key: eck.k8s.elastic.co/tenant + operator: In + values: + - team-a + - team-b + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + ingress: + - from: + - ipBlock: + cidr: 10.0.0.1/32 + ports: + - port: 9443 + protocol: TCP + podSelector: + matchLabels: + app.kubernetes.io/name: elastic-operator +``` + + +## Isolating Elasticsearch [k8s-network-policies-elasticsearch-isolation] + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9300 to other {{es}} nodes in the namespace (transport port).
* UDP port 53 for DNS lookup.
| +| Ingress (incoming) | * TCP port 9200 from the operator and other pods in the namespace.
* TCP port 9300 from other {{es}} nodes in the namespace (transport port).
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-elasticsearch + namespace: team-a +spec: + egress: + - ports: + - port: 9300 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + - ports: + - port: 53 + protocol: UDP + ingress: + - from: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/operator-name: elastic-operator + podSelector: + matchLabels: + app.kubernetes.io/name: elastic-operator + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + # [Optional] Allow ingress controller pods from the ingress-nginx namespace. + #- namespaceSelector: + # matchLabels: + # name: ingress-nginx + ports: + - port: 9200 + protocol: TCP + - from: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + ports: + - port: 9300 + protocol: TCP + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch +``` + + +## Isolating Kibana [k8s-network-policies-kibana-isolation] + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* UDP port 53 for DNS lookup.
| +| Ingress (incoming) | * TCP port 5601 from other pods in the namespace.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-kibana + namespace: team-a +spec: + egress: + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + # [Optional] Restrict to a single {es} cluster named hulk. + # elasticsearch.k8s.elastic.co/cluster-name=hulk + - ports: + - port: 53 + protocol: UDP + # [Optional] If Agent is deployed, this is to allow Kibana to access the Elastic Package Registry (https://epr.elastic.co). + # - port: 443 + # protocol: TCP + ingress: + - from: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + # [Optional] Allow ingress controller pods from the ingress-nginx namespace. + #- namespaceSelector: + # matchLabels: + # name: ingress-nginx + ports: + - port: 5601 + protocol: TCP + podSelector: + matchLabels: + common.k8s.elastic.co/type: kibana +``` + + +## Isolating APM Server [k8s-network-policies-apm-server-isolation] + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* UDP port 53 for DNS lookup.
| +| Ingress (incoming) | * TCP port 8200 from other pods in the namespace.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-apm-server + namespace: team-a +spec: + egress: + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + - ports: + - port: 5601 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: kibana + - ports: + - port: 53 + protocol: UDP + ingress: + - from: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + # [Optional] Allow ingress controller pods from the ingress-nginx namespace. + #- namespaceSelector: + # matchLabels: + # name: ingress-nginx + ports: + - port: 8200 + protocol: TCP + podSelector: + matchLabels: + common.k8s.elastic.co/type: apm-server +``` + +## Isolating Beats [k8s-network-policies-beats-isolation] + +::::{note} +Some {{beats}} may require additional access rules than what is listed here. For example, {{heartbeat}} will require a rule to allow access to the endpoint it is monitoring. +:::: + + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* UDP port 53 for DNS lookup.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-beats + namespace: team-a +spec: + egress: + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + - ports: + - port: 5601 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: kibana + - ports: + - port: 53 + protocol: UDP + podSelector: + matchLabels: + common.k8s.elastic.co/type: beat +``` + + +## Isolating Elastic Agent and Fleet [k8s-network-policies-agent-isolation] + +::::{note} +Some {{agent}} policies may require additional access rules other than those listed here. +:::: + + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* TCP port 8220 to {{fleet}} instances in the namespace.
* UDP port 53 for DNS lookup.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-agent + namespace: team-a +spec: + egress: + - ports: + - port: 8220 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: agent + - ports: + - port: 5601 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: kibana + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + - ports: + - port: 53 + protocol: UDP + - ports: + - port: 443 + protocol: TCP + to: + - ipBlock: + cidr: 10.0.0.1/32 + ingress: + - from: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + ports: + - port: 8220 + protocol: TCP + podSelector: + matchLabels: + common.k8s.elastic.co/type: agent +``` + +## Isolating Logstash [k8s-network-policies-logstash-isolation] + +::::{note} +{{ls}} may require additional access rules than those listed here, depending on plugin usage. +:::: + + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* UDP port 53 for DNS lookup.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-logstash + namespace: team-a +spec: + egress: + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + - ports: + - port: 53 + protocol: UDP + podSelector: + matchLabels: + common.k8s.elastic.co/type: logstash +``` diff --git a/deploy-manage/deploy/cloud-on-k8s/node-configuration.md b/deploy-manage/deploy/cloud-on-k8s/node-configuration.md index ece4a79187..68d572bedf 100644 --- a/deploy-manage/deploy/cloud-on-k8s/node-configuration.md +++ b/deploy-manage/deploy/cloud-on-k8s/node-configuration.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-node-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/nodes-orchestration.md b/deploy-manage/deploy/cloud-on-k8s/nodes-orchestration.md index 1c59c707ce..def6e10a8b 100644 --- a/deploy-manage/deploy/cloud-on-k8s/nodes-orchestration.md +++ b/deploy-manage/deploy/cloud-on-k8s/nodes-orchestration.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-orchestration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/orchestrate-other-elastic-applications.md b/deploy-manage/deploy/cloud-on-k8s/orchestrate-other-elastic-applications.md index 0ab9e93bda..2376f6ff08 100644 --- a/deploy-manage/deploy/cloud-on-k8s/orchestrate-other-elastic-applications.md +++ b/deploy-manage/deploy/cloud-on-k8s/orchestrate-other-elastic-applications.md @@ -1,3 +1,28 @@ # Orchestrate other Elastic applications -% What needs to be done: Write from scratch \ No newline at end of file +This section explains how to deploy and configure various Elastic Stack applications within Elastic Cloud on Kubernetes (ECK). + +::::{tip} +This content applies to APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash applications. To orchestrate an {{es}} cluster or {{kib}}, refer to [](./manage-deployments.md). +:::: + +The following guides provide specific instructions for deploying and configuring each application on ECK: +* [APM Server](apm-server.md) +* [Standalone Elastic Agent](standalone-elastic-agent.md) +* [{{fleet}}-managed {{agent}}](fleet-managed-elastic-agent.md) +* [Elastic Maps Server](elastic-maps-server.md) +* [Beats](beats.md) +* [{{ls}}](logstash.md) + +When orchestrating any of these applications, also consider the following topics: + +* [Elastic Stack Helm Chart](managing-deployments-using-helm-chart.md) +* [Recipes](recipes.md) +* [Secure the Elastic Stack](../../security.md) +* [Access Elastic Stack services](accessing-services.md) +* [Customize Pods](customize-pods.md) +* [Manage compute resources](manage-compute-resources.md) +* [Autoscaling stateless applications](../../autoscaling/autoscaling-stateless-applications-on-eck.md) +* [Elastic Stack configuration policies](elastic-stack-configuration-policies.md) +* [Upgrade the Elastic Stack version](../../upgrade/deployment-or-cluster.md) +* [Connect to external Elastic resources](connect-to-external-elastic-resources.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/pod-disruption-budget.md b/deploy-manage/deploy/cloud-on-k8s/pod-disruption-budget.md index 661b5f91f5..f19af44c6c 100644 --- a/deploy-manage/deploy/cloud-on-k8s/pod-disruption-budget.md +++ b/deploy-manage/deploy/cloud-on-k8s/pod-disruption-budget.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-pod-disruption-budget.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/pod-prestop-hook.md b/deploy-manage/deploy/cloud-on-k8s/pod-prestop-hook.md index 2609be706a..627bc57342 100644 --- a/deploy-manage/deploy/cloud-on-k8s/pod-prestop-hook.md +++ b/deploy-manage/deploy/cloud-on-k8s/pod-prestop-hook.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-prestop.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/quickstart-beats.md b/deploy-manage/deploy/cloud-on-k8s/quickstart-beats.md index bf98e7c60b..e55d5e6e58 100644 --- a/deploy-manage/deploy/cloud-on-k8s/quickstart-beats.md +++ b/deploy-manage/deploy/cloud-on-k8s/quickstart-beats.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat-quickstart.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/quickstart-fleet.md b/deploy-manage/deploy/cloud-on-k8s/quickstart-fleet.md index 809b99ac13..26b4423ee7 100644 --- a/deploy-manage/deploy/cloud-on-k8s/quickstart-fleet.md +++ b/deploy-manage/deploy/cloud-on-k8s/quickstart-fleet.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-fleet-quickstart.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/quickstart-logstash.md b/deploy-manage/deploy/cloud-on-k8s/quickstart-logstash.md index 396875ddef..e382b877ff 100644 --- a/deploy-manage/deploy/cloud-on-k8s/quickstart-logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/quickstart-logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-quickstart.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/quickstart-standalone.md b/deploy-manage/deploy/cloud-on-k8s/quickstart-standalone.md index e2144da968..8ec7c87b75 100644 --- a/deploy-manage/deploy/cloud-on-k8s/quickstart-standalone.md +++ b/deploy-manage/deploy/cloud-on-k8s/quickstart-standalone.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-quickstart.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/readiness-probe.md b/deploy-manage/deploy/cloud-on-k8s/readiness-probe.md index 93454872c4..ec7ef13f50 100644 --- a/deploy-manage/deploy/cloud-on-k8s/readiness-probe.md +++ b/deploy-manage/deploy/cloud-on-k8s/readiness-probe.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-readiness.html --- @@ -39,9 +41,10 @@ spec: Note that this requires restarting the Pods. - ## Elasticsearch versions 8.2.0 and later [k8s_elasticsearch_versions_8_2_0_and_later] +% this feature might have disappeared, we will need to investigate this a bit more, as the link below doesn't work anymore but it does for 8.15 for example. + We do not recommend overriding the default readiness probe on Elasticsearch 8.2.0 and later. ECK configures a socket based readiness probe using the Elasticsearch [readiness port feature](https://www.elastic.co/guide/en/elasticsearch/reference/current/advanced-configuration.html#readiness-tcp-port) which is not influenced by the load on the Elasticsearch cluster. diff --git a/deploy-manage/deploy/cloud-on-k8s/recipes.md b/deploy-manage/deploy/cloud-on-k8s/recipes.md index 3fd4028488..6d546cab13 100644 --- a/deploy-manage/deploy/cloud-on-k8s/recipes.md +++ b/deploy-manage/deploy/cloud-on-k8s/recipes.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-recipes.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md b/deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md index 1b5ac483be..79cc20359c 100644 --- a/deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md +++ b/deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-traffic-splitting.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/required-rbac-permissions.md b/deploy-manage/deploy/cloud-on-k8s/required-rbac-permissions.md index 27bedd92d0..6ff4dceb6a 100644 --- a/deploy-manage/deploy/cloud-on-k8s/required-rbac-permissions.md +++ b/deploy-manage/deploy/cloud-on-k8s/required-rbac-permissions.md @@ -1,11 +1,13 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-eck-permissions.html --- # Required RBAC permissions [k8s-eck-permissions] -Installing and running ECK, as well as using ECK-managed resources requires the following Kubernetes [permissions](https://kubernetes.io/docs/reference/access-authn-authz/rbac): +Installing and running ECK, as well as using ECK-managed resources, requires the following Kubernetes [permissions](https://kubernetes.io/docs/reference/access-authn-authz/rbac): * [Installing CRDs](#k8s-eck-permissions-installing-crds) * [Installing the ECK operator](#k8s-eck-permissions-installing-operator) diff --git a/deploy-manage/deploy/cloud-on-k8s/restrict-cross-namespace-resource-associations.md b/deploy-manage/deploy/cloud-on-k8s/restrict-cross-namespace-resource-associations.md index ddbf133772..60b96bbdad 100644 --- a/deploy-manage/deploy/cloud-on-k8s/restrict-cross-namespace-resource-associations.md +++ b/deploy-manage/deploy/cloud-on-k8s/restrict-cross-namespace-resource-associations.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-restrict-cross-namespace-associations.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/securing-logstash-api.md b/deploy-manage/deploy/cloud-on-k8s/securing-logstash-api.md index 421346f41f..2de0724b61 100644 --- a/deploy-manage/deploy/cloud-on-k8s/securing-logstash-api.md +++ b/deploy-manage/deploy/cloud-on-k8s/securing-logstash-api.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-securing-api.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/security-context.md b/deploy-manage/deploy/cloud-on-k8s/security-context.md index cc8460d7c1..3abb59085b 100644 --- a/deploy-manage/deploy/cloud-on-k8s/security-context.md +++ b/deploy-manage/deploy/cloud-on-k8s/security-context.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-security-context.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/service-meshes.md b/deploy-manage/deploy/cloud-on-k8s/service-meshes.md index 1c943134ab..4530b8dd38 100644 --- a/deploy-manage/deploy/cloud-on-k8s/service-meshes.md +++ b/deploy-manage/deploy/cloud-on-k8s/service-meshes.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-service-meshes.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/settings-managed-by-eck.md b/deploy-manage/deploy/cloud-on-k8s/settings-managed-by-eck.md index 58b6e22bbc..617935e8b2 100644 --- a/deploy-manage/deploy/cloud-on-k8s/settings-managed-by-eck.md +++ b/deploy-manage/deploy/cloud-on-k8s/settings-managed-by-eck.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-reserved-settings.html --- @@ -10,8 +12,8 @@ The following Elasticsearch settings are managed by ECK: * `cluster.name` * `discovery.seed_hosts` * `discovery.seed_providers` -* `discovery.zen.minimum_master_nodes` [7.0] -* `cluster.initial_master_nodes` [7.0] +* `discovery.zen.minimum_master_nodes` +* `cluster.initial_master_nodes` * `network.host` * `network.publish_host` * `path.data` diff --git a/deploy-manage/deploy/cloud-on-k8s/standalone-elastic-agent.md b/deploy-manage/deploy/cloud-on-k8s/standalone-elastic-agent.md index 7f597dec22..dfccd861ef 100644 --- a/deploy-manage/deploy/cloud-on-k8s/standalone-elastic-agent.md +++ b/deploy-manage/deploy/cloud-on-k8s/standalone-elastic-agent.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/storage-recommendations.md b/deploy-manage/deploy/cloud-on-k8s/storage-recommendations.md index 0505593e02..2a3f64f979 100644 --- a/deploy-manage/deploy/cloud-on-k8s/storage-recommendations.md +++ b/deploy-manage/deploy/cloud-on-k8s/storage-recommendations.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-storage-recommendations.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/tls-certificates.md b/deploy-manage/deploy/cloud-on-k8s/tls-certificates.md deleted file mode 100644 index 324713c7b7..0000000000 --- a/deploy-manage/deploy/cloud-on-k8s/tls-certificates.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-tls-certificates.html ---- - -# TLS Certificates [k8s-tls-certificates] - -This section only covers TLS certificates for the HTTP layer. TLS certificates for the transport layer that are used for internal communications between Elasticsearch nodes are managed by ECK and cannot be changed. You can however set your own certificate authority for the [transport layer](transport-settings.md#k8s-transport-ca). - -## Default self-signed certificate [k8s-default-self-signed-certificate] - -By default, the operator manages a self-signed certificate with a custom CA for each resource. The CA, the certificate and the private key are each stored in a separate `Secret`. - -```sh -> kubectl get secret | grep es-http -hulk-es-http-ca-internal Opaque 2 28m -hulk-es-http-certs-internal Opaque 2 28m -hulk-es-http-certs-public Opaque 1 28m -``` - -The public certificate is stored in a secret named `-[es|kb|apm|ent|agent]-http-certs-public`. - -```sh -> kubectl get secret hulk-es-http-certs-public -o go-template='{{index .data "tls.crt" | base64decode }}' ------BEGIN CERTIFICATE----- -MIIDQDCCAiigAwIBAgIQHC4O/RWX15a3/P3upsm3djANBgkqhkiG9w0BAQsFADA6 -... -QLYL4zLEby3vRxq65+xofVBJAaM= ------END CERTIFICATE----- -``` - -### Reserve static IP and custom domain [k8s-static-ip-custom-domain] - -To use a custom domain name with the self-signed certificate, you can reserve a static IP and/or use an Ingress instead of a `LoadBalancer` `Service`. Whatever you use, your DNS must be added to the certificate SAN in the `spec.http.tls.selfSignedCertificate.subjectAltNames` section of your Elastic resource manifest. - -```yaml -spec: - http: - service: - spec: - type: LoadBalancer - tls: - selfSignedCertificate: - subjectAltNames: - - ip: 160.46.176.15 - - dns: hulk.example.com -``` - - - -## Setup your own certificate [k8s-setting-up-your-own-certificate] - -You can bring your own certificate to configure TLS to ensure that communication between HTTP clients and the Elastic Stack application is encrypted. - -Create a Kubernetes secret with: - -* `ca.crt`: CA certificate (optional if `tls.crt` was issued by a well-known CA). -* `tls.crt`: The certificate. -* `tls.key`: The private key to the first certificate in the certificate chain. - -::::{warning} -If your `tls.crt` is signed by an intermediate CA you may need both the Root CA and the intermediate CA combined within the `ca.crt` file depending on whether the Root CA is globally trusted. -:::: - - -```sh -kubectl create secret generic my-cert --from-file=ca.crt --from-file=tls.crt --from-file=tls.key -``` - -Alternatively you can also bring your own CA certificate including a private key and let ECK issue certificates with it. Any certificate SANs you have configured as decribed in [Reserve static IP and custom domain](#k8s-static-ip-custom-domain) will also be respected when issuing certificates with this CA certificate. - -Create a Kubernetes secret with: - -* `ca.crt`: CA certificate. -* `ca.key`: The private key to the CA certificate. - -```sh -kubectl create secret generic my-cert --from-file=ca.crt --from-file=ca.key -``` - -In both cases, you have to reference the secret name in the `http.tls.certificate` section of the resource manifest. - -```yaml -spec: - http: - tls: - certificate: - secretName: my-cert -``` - - -## Disable TLS [k8s-disable-tls] - -You can explicitly disable TLS for Kibana, APM Server, and the HTTP layer of Elasticsearch. - -```yaml -spec: - http: - tls: - selfSignedCertificate: - disabled: true -``` - - diff --git a/deploy-manage/deploy/cloud-on-k8s/transport-settings.md b/deploy-manage/deploy/cloud-on-k8s/transport-settings.md index 32a5dea1bd..37ff1f3ba3 100644 --- a/deploy-manage/deploy/cloud-on-k8s/transport-settings.md +++ b/deploy-manage/deploy/cloud-on-k8s/transport-settings.md @@ -1,11 +1,13 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-transport-settings.html --- # Transport settings [k8s-transport-settings] -The transport module in Elasticsearch is used for internal communication between nodes within the cluster as well as communication between remote clusters. Check the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html) for details. For customization options of the HTTP layer, check [Services](accessing-services.md) and [TLS certificates](tls-certificates.md). +The transport module in Elasticsearch is used for internal communication between nodes within the cluster as well as communication between remote clusters. Check the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html) for details. For customization options of the HTTP layer, check [Services](accessing-services.md) and [TLS certificates](/deploy-manage/security/secure-http-communications.md). ## Customize the Transport Service [k8s_customize_the_transport_service] @@ -29,7 +31,6 @@ When you change the `clusterIP` setting of the service, ECK deletes and re-creat :::: - ## Configure a custom Certificate Authority [k8s-transport-ca] Elasticsearch uses X.509 certificates to establish encrypted and authenticated connections across nodes in the cluster. By default, ECK creates a self-signed CA certificate to issue a certificate [for each node in the cluster](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup.html#encrypt-internode-communication). @@ -120,8 +121,6 @@ spec: 3. If the remote cluster server is enabled, then the DNS names must also include both:* The DNS name for the related Kubernetes `Service`: `-es-remote-cluster.${POD_NAMESPACE}.svc` * The Pod DNS name: `${POD_NAME}.-es-.${POD_NAMESPACE}.svc` - - The following manifest is only provided to illustrate how these certificates can be configured in principle, using the trust-manager Bundle resource and cert-manager provisioned certificates: ```yaml diff --git a/deploy-manage/deploy/cloud-on-k8s/troubleshooting-beats.md b/deploy-manage/deploy/cloud-on-k8s/troubleshooting-beats.md index df5a2f81aa..b395f520e1 100644 --- a/deploy-manage/deploy/cloud-on-k8s/troubleshooting-beats.md +++ b/deploy-manage/deploy/cloud-on-k8s/troubleshooting-beats.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat-troubleshooting.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/update-deployments.md b/deploy-manage/deploy/cloud-on-k8s/update-deployments.md index a0f05f43e9..8ce6fabaac 100644 --- a/deploy-manage/deploy/cloud-on-k8s/update-deployments.md +++ b/deploy-manage/deploy/cloud-on-k8s/update-deployments.md @@ -1,4 +1,7 @@ --- +navigation_title: Applying updates +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-update-deployment.html --- @@ -25,7 +28,4 @@ spec: EOF ``` -ECK will automatically schedule the requested update. Changes can be monitored with the [ECK operator logs](install-using-yaml-manifest-quickstart.md), [`events`](https://kubernetes.io/docs/reference/kubernetes-api/cluster-resources/event-v1/), and applicable product’s [pod `logs`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_logs/). These will either report successful application of changes or provide context for further troubleshooting. Kindly note, Kubernetes restricts some changes, for example refer to [Updating Volume Claims](volume-claim-templates.md#k8s-volume-claim-templates-update). - -This completes our quickstart guide for deploying an {{es}} cluster and {{kib}} instance with our ECK operator. We recommend continuing to [Orchestrating Elastic Stack applications](configure-deployments.md) for more configuration options - +ECK will automatically schedule the requested update. Changes can be monitored with the [ECK operator logs](install-using-yaml-manifest-quickstart.md), [`events`](https://kubernetes.io/docs/reference/kubernetes-api/cluster-resources/event-v1/), and applicable product’s [pod `logs`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_logs/). These will either report successful application of changes or provide context for further troubleshooting. Kindly note, Kubernetes restricts some changes, for example refer to [Updating Volume Claims](volume-claim-templates.md#k8s-volume-claim-templates-update). \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/update-strategy-logstash.md b/deploy-manage/deploy/cloud-on-k8s/update-strategy-logstash.md index 574908c0dd..6a93577feb 100644 --- a/deploy-manage/deploy/cloud-on-k8s/update-strategy-logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/update-strategy-logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-update-strategy.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/update-strategy.md b/deploy-manage/deploy/cloud-on-k8s/update-strategy.md index decd1f262e..09d1c393f2 100644 --- a/deploy-manage/deploy/cloud-on-k8s/update-strategy.md +++ b/deploy-manage/deploy/cloud-on-k8s/update-strategy.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-update-strategy.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/use-an-elasticsearch-cluster-managed-by-eck.md b/deploy-manage/deploy/cloud-on-k8s/use-an-elasticsearch-cluster-managed-by-eck.md index a4a93e0aa7..8810ca43e3 100644 --- a/deploy-manage/deploy/cloud-on-k8s/use-an-elasticsearch-cluster-managed-by-eck.md +++ b/deploy-manage/deploy/cloud-on-k8s/use-an-elasticsearch-cluster-managed-by-eck.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-eck-managed-es.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/virtual-memory.md b/deploy-manage/deploy/cloud-on-k8s/virtual-memory.md index 52ebf84dbc..b8b11dffb8 100644 --- a/deploy-manage/deploy/cloud-on-k8s/virtual-memory.md +++ b/deploy-manage/deploy/cloud-on-k8s/virtual-memory.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-virtual-memory.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md b/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md index 0b713c23dc..fa18192283 100644 --- a/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md +++ b/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-volume-claim-templates.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/webhook-namespace-selectors.md b/deploy-manage/deploy/cloud-on-k8s/webhook-namespace-selectors.md index 41e052ff26..661a32e7ef 100644 --- a/deploy-manage/deploy/cloud-on-k8s/webhook-namespace-selectors.md +++ b/deploy-manage/deploy/cloud-on-k8s/webhook-namespace-selectors.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-webhook-namespace-selectors.html --- @@ -15,6 +17,5 @@ Webhook resources are cluster-scoped, therefore `createClusterScopedResources` m It is not recommended to deploy webhook resources in environments where operators are run by untrusted users and need to be locked down tightly. :::: - For more information, check [Configure the validating webhook](configure-validating-webhook.md) and [Dynamic Admission Control](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/). diff --git a/deploy-manage/license/manage-your-license-in-eck.md b/deploy-manage/license/manage-your-license-in-eck.md index 3cd98799a8..a1fd8687e6 100644 --- a/deploy-manage/license/manage-your-license-in-eck.md +++ b/deploy-manage/license/manage-your-license-in-eck.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-licensing.html --- diff --git a/deploy-manage/security/secure-http-communications.md b/deploy-manage/security/secure-http-communications.md index 933f4a529e..43675d22b8 100644 --- a/deploy-manage/security/secure-http-communications.md +++ b/deploy-manage/security/secure-http-communications.md @@ -20,6 +20,9 @@ mapped_urls: % - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-custom-http-certificate.md % - [ ] ./raw-migrated-files/kibana/kibana/Security-production-considerations.md +% EEDUGON NOTE: security section might miss a section to secure the transport layer (not the HTTP). +% There we should integrate the content of https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-transport-settings.html which is currently in ECK (/deploy-manage) doc. + % Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): $$$encrypt-kibana-browser$$$ @@ -46,4 +49,6 @@ $$$csp-strict-mode$$$ $$$k8s-setting-up-your-own-certificate$$$ -$$$k8s-static-ip-custom-domain$$$ \ No newline at end of file +$$$k8s-static-ip-custom-domain$$$ + +$$$k8s-disable-tls$$$ \ No newline at end of file diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index 3224f254fa..15ef9b9e0e 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -232,6 +232,7 @@ toc: children: - file: deploy/cloud-on-k8s/install-using-yaml-manifest-quickstart.md - file: deploy/cloud-on-k8s/install-using-helm-chart.md + - file: deploy/cloud-on-k8s/required-rbac-permissions.md - file: deploy/cloud-on-k8s/deploy-eck-on-openshift.md children: - file: deploy/cloud-on-k8s/k8s-openshift-deploy-operator.md @@ -241,27 +242,18 @@ toc: - file: deploy/cloud-on-k8s/k8s-openshift-beats.md - file: deploy/cloud-on-k8s/k8s-openshift-agent.md - file: deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md - children: - - file: deploy/cloud-on-k8s/k8s-autopilot-setting-virtual-memory.md - - file: deploy/cloud-on-k8s/k8s-autopilot-deploy-operator.md - - file: deploy/cloud-on-k8s/k8s-autopilot-deploy-elasticsearch.md - - file: deploy/cloud-on-k8s/k8s-autopilot-deploy-agent-beats.md - file: deploy/cloud-on-k8s/deploy-fips-compatible-version-of-eck.md - file: deploy/cloud-on-k8s/air-gapped-install.md - file: deploy/cloud-on-k8s/configure.md children: - file: deploy/cloud-on-k8s/configure-eck.md - - file: deploy/cloud-on-k8s/required-rbac-permissions.md - file: deploy/cloud-on-k8s/configure-validating-webhook.md - file: deploy/cloud-on-k8s/restrict-cross-namespace-resource-associations.md - - file: deploy/cloud-on-k8s/create-custom-images.md - file: deploy/cloud-on-k8s/service-meshes.md children: - file: deploy/cloud-on-k8s/k8s-service-mesh-istio.md - file: deploy/cloud-on-k8s/k8s-service-mesh-linkerd.md - file: deploy/cloud-on-k8s/network-policies.md - children: - - file: deploy/cloud-on-k8s/k8s_prerequisites.md - file: deploy/cloud-on-k8s/webhook-namespace-selectors.md - file: deploy/cloud-on-k8s/manage-deployments.md children: @@ -274,21 +266,22 @@ toc: children: - file: deploy/cloud-on-k8s/elasticsearch-configuration.md children: + - file: deploy/cloud-on-k8s/nodes-orchestration.md + - file: deploy/cloud-on-k8s/storage-recommendations.md - file: deploy/cloud-on-k8s/node-configuration.md - file: deploy/cloud-on-k8s/volume-claim-templates.md - - file: deploy/cloud-on-k8s/storage-recommendations.md - - file: deploy/cloud-on-k8s/transport-settings.md - file: deploy/cloud-on-k8s/virtual-memory.md - file: deploy/cloud-on-k8s/settings-managed-by-eck.md - file: deploy/cloud-on-k8s/custom-configuration-files-plugins.md - file: deploy/cloud-on-k8s/init-containers-for-plugin-downloads.md + - file: deploy/cloud-on-k8s/transport-settings.md - file: deploy/cloud-on-k8s/update-strategy.md - file: deploy/cloud-on-k8s/pod-disruption-budget.md - - file: deploy/cloud-on-k8s/nodes-orchestration.md - file: deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md - file: deploy/cloud-on-k8s/readiness-probe.md - file: deploy/cloud-on-k8s/pod-prestop-hook.md - file: deploy/cloud-on-k8s/security-context.md + - file: deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md - file: deploy/cloud-on-k8s/kibana-configuration.md children: - file: deploy/cloud-on-k8s/k8s-kibana-es.md @@ -296,13 +289,11 @@ toc: - file: deploy/cloud-on-k8s/k8s-kibana-secure-settings.md - file: deploy/cloud-on-k8s/k8s-kibana-http-configuration.md - file: deploy/cloud-on-k8s/k8s-kibana-plugins.md - - file: deploy/cloud-on-k8s/tls-certificates.md - - file: deploy/cloud-on-k8s/recipes.md - - file: deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md - file: deploy/cloud-on-k8s/customize-pods.md - file: deploy/cloud-on-k8s/manage-compute-resources.md - - file: deploy/cloud-on-k8s/elastic-stack-configuration-policies.md + - file: deploy/cloud-on-k8s/recipes.md - file: deploy/cloud-on-k8s/connect-to-external-elastic-resources.md + - file: deploy/cloud-on-k8s/elastic-stack-configuration-policies.md - file: deploy/cloud-on-k8s/orchestrate-other-elastic-applications.md children: - file: deploy/cloud-on-k8s/apm-server.md @@ -342,6 +333,7 @@ toc: - file: deploy/cloud-on-k8s/configuration-examples-logstash.md - file: deploy/cloud-on-k8s/update-strategy-logstash.md - file: deploy/cloud-on-k8s/advanced-configuration-logstash.md + - file: deploy/cloud-on-k8s/create-custom-images.md - file: deploy/cloud-on-k8s/tools-apis.md - file: deploy/self-managed.md children: @@ -855,4 +847,4 @@ toc: - file: uninstall/uninstall-elastic-cloud-enterprise.md - file: uninstall/uninstall-elastic-cloud-on-kubernetes.md - file: uninstall/uninstall-a-self-managed-cluster.md - - file: uninstall/delete-a-cloud-deployment.md \ No newline at end of file + - file: uninstall/delete-a-cloud-deployment.md diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md deleted file mode 100644 index 280cfd2ae1..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md +++ /dev/null @@ -1,19 +0,0 @@ -# Access Elastic Stack services [k8s-accessing-elastic-services] - -To access the Elastic Stack services, you need to: - -* Retrieve the `elastic` user password for basic authentication. -* Specify the IP of the service, if you want to access the service from outside the Kubernetes cluster. -* Decide if you want to use the self-signed certificate generated by ECK, or configure your own certificate. - -The following sections will guide you through this process: - -* [Security](../../../deploy-manage/security/secure-cluster-communications.md) -* [Services](../../../deploy-manage/deploy/cloud-on-k8s/accessing-services.md) -* [TLS certificates](../../../deploy-manage/security/secure-http-communications.md) -* [Access the Elasticsearch endpoint](../../../deploy-manage/deploy/cloud-on-k8s/accessing-services.md) - - - - - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md deleted file mode 100644 index 227a31684b..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md +++ /dev/null @@ -1,12 +0,0 @@ -# Advanced topics [k8s-advanced-topics] - -* [*Deploy ECK on OpenShift*](../../../deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-openshift.md) -* [*Deploy ECK on GKE Autopilot*](../../../deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md) -* [*Create custom images*](../../../deploy-manage/deploy/cloud-on-k8s/create-custom-images.md) -* [*Service meshes*](../../../deploy-manage/deploy/cloud-on-k8s/service-meshes.md) -* [*Traffic Splitting*](../../../deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md) -* [*Network policies*](../../../deploy-manage/deploy/cloud-on-k8s/network-policies.md) -* [*Webhook namespace selectors*](../../../deploy-manage/deploy/cloud-on-k8s/webhook-namespace-selectors.md) -* [*Stack Monitoring*](../../../deploy-manage/monitor/stack-monitoring/eck-stack-monitoring.md) -* [*Deploy a FIPS compatible version of ECK*](../../../deploy-manage/deploy/cloud-on-k8s/deploy-fips-compatible-version-of-eck.md) - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-air-gapped.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-air-gapped.md deleted file mode 100644 index da626e0ff6..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-air-gapped.md +++ /dev/null @@ -1,65 +0,0 @@ -# Running in air-gapped environments [k8s-air-gapped] - -The ECK operator can be run in an air-gapped environment without access to the open internet when it is configured not to pull container images from `docker.elastic.co`. - -By default ECK does not require you to specify the container image for each Elastic Stack application you deploy. - -```yaml -apiVersion: elasticsearch.k8s.elastic.co/v1 -kind: Elasticsearch -metadata: - name: quickstart -spec: - version: 8.16.1 - # image: docker.elastic.co/elasticsearch/elasticsearch:8.16.1 <1> - nodeSets: - - name: default - count: 1 - # podTemplate: - # spec: - # imagePullSecrets: <2> - # - name: private-registry-credentials-secret -``` - -1. The ECK operator will set this value by default. You can explicitly set it to your mirrored container image when running in an air-gapped environment -2. You can provide credentials to your private container registry by setting the `imagePullSecrets` field through the `spec.podTemplate` section of your Elastic resource specification, check [how to customize the Elastic resources Pods](../../../deploy-manage/deploy/cloud-on-k8s/customize-pods.md) and [how to setup a Secret containing your registry credentials](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). - - -ECK will automatically set the correct container image for each application. When running in an air-gapped or offline environment you will have to mirror the official Elastic container images in a private container image registry. To make use of your mirrored images you can either set the image for each application explicitly as shown in the preceding example or more conveniently override the default container registry as explained in the next section. - - -## Use a mirrored image of the ECK operator [k8s-use-mirrored-operator-image] - -To deploy the ECK operator in an air-gapped environment, you first have to mirror the operator image itself from `docker.elastic.co` to a private container registry, for example `my.registry`. - -Once the ECK operator image is copied internally, replace the original image name `docker.elastic.co/eck/eck-operator:2.16.1` with the private name of the image, for example `my.registry/eck/eck-operator:2.16.1`, in the [operator manifests](../../../deploy-manage/deploy/cloud-on-k8s/install-using-yaml-manifest-quickstart.md). When using [Helm charts](../../../deploy-manage/deploy/cloud-on-k8s/install-using-helm-chart.md), replace the `image.repository` Helm value with, for example, `my.registry/eck/eck-operator`. - - -## Override the default container registry [k8s-container-registry-override] - -When creating custom resources (Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash), the operator defaults to using container images pulled from the `docker.elastic.co` registry. If you are in an environment where external network access is restricted, you can configure the operator to use a different default container registry by starting the operator with the `--container-registry` command-line flag. Check [*Configure ECK*](../../../deploy-manage/deploy/cloud-on-k8s/configure-eck.md) for more information on how to configure the operator using command-line flags and environment variables. - -The operator expects container images to be located at specific repositories in the default container registry. Make sure that your container images are stored in the right repositories and are tagged correctly with the Stack version number. For example, if your private registry is `my.registry` and you wish to deploy components from Stack version 8.16.1, the following image names should exist: - -* `my.registry/elasticsearch/elasticsearch:8.16.1` -* `my.registry/kibana/kibana:8.16.1` -* `my.registry/apm/apm-server:8.16.1` - - -## Use a global container repository [k8s-container-repository-override] - -If you cannot follow the default Elastic image repositories naming scheme, you can configure the operator to use a different container repository by starting the operator with the `--container-repository` command-line flag. Check [*Configure ECK*](../../../deploy-manage/deploy/cloud-on-k8s/configure-eck.md) for more information on how to configure the operator using command-line flags and environment variables. - -For example, if your private registry is `my.registry` and all Elastic images are located under the `elastic` repository, the following image names should exist: - -* `my.registry/elastic/elasticsearch:8.16.1` -* `my.registry/elastic/kibana:8.16.1` -* `my.registry/elastic/apm-server:8.16.1` - - -## ECK Diagnostics in air-gapped environments [k8s-eck-diag-air-gapped] - -The [eck-diagnostics tool](../../../troubleshoot/deployments/cloud-on-k8s/run-eck-diagnostics.md) optionally runs diagnostics for Elastic Stack applications in a separate container that is deployed into the Kubernetes cluster. - -In air-gapped environments with no access to the `docker.elastic.co` registry, you should copy the latest support-diagnostics container image to your internal image registry and then run the tool with the additional flag `--diagnostic-image `. To find out which support diagnostics container image matches your version of eck-diagnostics run the tool once without arguments and it will print the default image in use. - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-deploy-eck.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-deploy-eck.md deleted file mode 100644 index fa2966fd2b..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-deploy-eck.md +++ /dev/null @@ -1,57 +0,0 @@ -# Deploy ECK in your Kubernetes cluster [k8s-deploy-eck] - -Things to consider before you start: - -* For this quickstart guide, your Kubernetes cluster is assumed to be already up and running. Before you proceed with the ECK installation, make sure you check the [supported versions](../../../deploy-manage/deploy/cloud-on-k8s.md). -* If you are using GKE, make sure your user has `cluster-admin` permissions. For more information, check [Prerequisites for using Kubernetes RBAC on GKE](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control#iam-rolebinding-bootstrap). -* If you are using Amazon EKS, make sure the Kubernetes control plane is allowed to communicate with the Kubernetes nodes on port 443. This is required for communication with the Validating Webhook. For more information, check [Recommended inbound traffic](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.md). -* Refer to [*Install ECK*](../../../deploy-manage/deploy/cloud-on-k8s/install.md) for more information on installation options. -* Check the [upgrade notes](../../../deploy-manage/upgrade/orchestrator/upgrade-cloud-on-k8s.md) if you are attempting to upgrade an existing ECK deployment. - -To deploy the ECK operator: - -1. Install [custom resource definitions](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) with [`create`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_create/): - - ```sh - kubectl create -f https://download.elastic.co/downloads/eck/2.16.1/crds.yaml - ``` - - This will output similar to the following upon Elastic resources' creation: - - ```sh - customresourcedefinition.apiextensions.k8s.io/agents.agent.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/apmservers.apm.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/beats.beat.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/elasticmapsservers.maps.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/elasticsearches.elasticsearch.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/enterprisesearches.enterprisesearch.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/kibanas.kibana.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/logstashes.logstash.k8s.elastic.co created - ``` - -2. Install the operator with its RBAC rules with [`apply`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_apply/): - - ```sh - kubectl apply -f https://download.elastic.co/downloads/eck/2.16.1/operator.yaml - ``` - - ::::{note} - The ECK operator runs by default in the `elastic-system` namespace. It is recommended that you choose a dedicated namespace for your workloads, rather than using the `elastic-system` or the `default` namespace. - :::: - -3. Monitor the operator’s setup from its logs through [`logs`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_logs/): - - ```sh - kubectl -n elastic-system logs -f statefulset.apps/elastic-operator - ``` - -4. Once ready, the operator will report as `Running` as shown with [`get`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_get/), replacing default `elastic-system` with applicable installation namespace as needed: * - -``` -$ kubectl get -n elastic-system pods -NAME READY STATUS RESTARTS AGE -elastic-operator-0 1/1 Running 0 1m -``` - -This completes the quickstart of the ECK operator. We recommend continuing to [Deploying an {{es}} cluster](../../../deploy-manage/deploy/cloud-on-k8s/elasticsearch-deployment-quickstart.md); but for more configuration options as needed, navigate to [Operating ECK](../../../deploy-manage/deploy/cloud-on-k8s/configure.md). - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-fips.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-fips.md deleted file mode 100644 index c861994cee..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-fips.md +++ /dev/null @@ -1,10 +0,0 @@ -# Deploy a FIPS compatible version of ECK [k8s-fips] - -The Federal Information Processing Standard (FIPS) Publication 140-2, (FIPS PUB 140-2), titled "Security Requirements for Cryptographic Modules" is a U.S. government computer security standard used to approve cryptographic modules. Since version 2.6 ECK offers a FIPS-enabled image that is a drop-in replacement for the standard image. - -For the ECK operator, adherence to FIPS 140-2 is ensured by: - -* Using FIPS approved / NIST recommended cryptographic algorithms. -* Compiling the operator using the [BoringCrypto](https://github.com/golang/go/blob/dev.boringcrypto/README.boringcrypto.md) library for various cryptographic primitives. - - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-install-yaml-manifests.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-install-yaml-manifests.md deleted file mode 100644 index 89840c3781..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-install-yaml-manifests.md +++ /dev/null @@ -1,10 +0,0 @@ -# Install ECK using the YAML manifests [k8s-install-yaml-manifests] - -This method is the quickest way to get started with ECK if you have full administrative access to the Kubernetes cluster. The [Quickstart](../../../deploy-manage/deploy/cloud-on-k8s/deploy-an-orchestrator.md) document describes how to proceed with this method. When you run the `kubectl` command listed in [*Deploy ECK in your Kubernetes cluster*](../../../deploy-manage/deploy/cloud-on-k8s/install-using-yaml-manifest-quickstart.md), the following components are installed or updated: - -* `CustomResourceDefinition` objects for all supported resource types (Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash). -* `Namespace` named `elastic-system` to hold all operator resources. -* `ServiceAccount`, `ClusterRole` and `ClusterRoleBinding` to allow the operator to manage resources throughout the cluster. -* `ValidatingWebhookConfiguration` to validate Elastic custom resources on admission. -* `StatefulSet`, `ConfigMap`, `Secret` and `Service` in `elastic-system` namespace to run the operator application. - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-overview.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-overview.md deleted file mode 100644 index 69d1ad6b43..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-overview.md +++ /dev/null @@ -1,12 +0,0 @@ -# Overview [k8s-overview] - -Built on the Kubernetes Operator pattern, Elastic Cloud on Kubernetes (ECK) extends the basic Kubernetes orchestration capabilities to support the setup and management of Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash on Kubernetes. - -With Elastic Cloud on Kubernetes you can streamline critical operations, such as: - -1. Managing and monitoring multiple clusters -2. Scaling cluster capacity and storage -3. Performing safe configuration changes through rolling upgrades -4. Securing clusters with TLS certificates -5. Setting up hot-warm-cold architectures with availability zone awareness - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md deleted file mode 100644 index c73311711a..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md +++ /dev/null @@ -1,34 +0,0 @@ -# Access the Elasticsearch endpoint [k8s-request-elasticsearch-endpoint] - -You can access the Elasticsearch endpoint within or outside the Kubernetes cluster. - -**Within the Kubernetes cluster** - -1. Retrieve the CA certificate. -2. Retrieve the password of the `elastic` user. - -```sh -NAME=hulk - -kubectl get secret "$NAME-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt -PW=$(kubectl get secret "$NAME-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}') - -curl --cacert tls.crt -u elastic:$PW https://$NAME-es-http:9200/ -``` - -**Outside the Kubernetes cluster** - -1. Retrieve the CA certificate. -2. Retrieve the password of the `elastic` user. -3. Retrieve the IP of the `LoadBalancer` `Service`. - -```sh -NAME=hulk - -kubectl get secret "$NAME-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt -IP=$(kubectl get svc "$NAME-es-http" -o jsonpath='{.status.loadBalancer.ingress[].ip}') -PW=$(kubectl get secret "$NAME-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}') - -curl --cacert tls.crt -u elastic:$PW https://$IP:9200/ -``` - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-security.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-security.md deleted file mode 100644 index 068c54755f..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-security.md +++ /dev/null @@ -1,19 +0,0 @@ -# Security [k8s-security] - -All Elastic Stack resources deployed by the ECK operator are secured by default. The operator sets up basic authentication and TLS to encrypt network traffic to, from, and within your Elasticsearch cluster. - -## Authentication [k8s-authentication] - -To access Elastic resources, the operator manages a default user named `elastic` with the `superuser` role. Its password is stored in a `Secret` named `-elastic-user`. - -```sh -> kubectl get secret hulk-es-elastic-user -o go-template='{{.data.elastic | base64decode }}' -42xyz42citsale42xyz42 -``` - -::::{note} -Beware of copying this Secret as-is into a different namespace. Check [Common Problems: Owner References](../../../troubleshoot/deployments/cloud-on-k8s/common-problems.md#k8s-common-problems-owner-refs) for more information. -:::: - - - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-services.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-services.md deleted file mode 100644 index f4a1791f2d..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-services.md +++ /dev/null @@ -1,50 +0,0 @@ -# Services [k8s-services] - -You can access Elastic resources by using native Kubernetes services that are not reachable from the public Internet by default. - -## Manage Kubernetes services [k8s-kubernetes-service] - -For each resource, the operator manages a Kubernetes service named `-[es|kb|apm|ent|agent]-http`, which is of type `ClusterIP` by default. `ClusterIP` exposes the service on a cluster-internal IP and makes the service only reachable from the cluster. - -```sh -> kubectl get svc - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -hulk-apm-http ClusterIP 10.19.212.105 8200/TCP 1m -hulk-es-http ClusterIP 10.19.252.160 9200/TCP 1m -hulk-kb-http ClusterIP 10.19.247.151 5601/TCP 1m -``` - - -## Allow public access [k8s-allow-public-access] - -You can expose services in [different ways](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types) by specifying an `http.service.spec.type` in the `spec` of the resource manifest. On cloud providers which support external load balancers, you can set the `type` field to `LoadBalancer` to provision a load balancer for the `Service`, and populate the column `EXTERNAL-IP` after a short delay. Depending on the cloud provider, it may incur costs. - -By default, the Elasticsearch service created by ECK is configured to route traffic to all Elasticsearch nodes in the cluster. Depending on your cluster configuration, you may want more control over the set of nodes that handle different types of traffic (query, ingest, and so on). Check [*Traffic Splitting*](../../../deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md) for more information. - -::::{warning} -When you change the `clusterIP` setting of the service, ECK will delete and re-create the service as `clusterIP` is an immutable field. Depending on your client implementation, this might result in a short disruption until the service DNS entries refresh to point to the new endpoints. -:::: - - -```yaml -apiVersion: .k8s.elastic.co/v1 -kind: -metadata: - name: hulk -spec: - version: 8.16.1 - http: - service: - spec: - type: LoadBalancer -``` - -```sh -> kubectl get svc - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -hulk-apm-http LoadBalancer 10.19.212.105 35.176.227.106 8200:31000/TCP 1m -hulk-es-http LoadBalancer 10.19.252.160 35.198.131.115 9200:31320/TCP 1m -hulk-kb-http LoadBalancer 10.19.247.151 35.242.197.228 5601:31380/TCP 1m -``` diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-supported.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-supported.md deleted file mode 100644 index 58850c32e2..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-supported.md +++ /dev/null @@ -1,24 +0,0 @@ -# Supported versions [k8s-supported] - -ECK is compatible with: - -* Kubernetes 1.28-1.32 -* OpenShift 4.12-4.17 -* Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), and Amazon Elastic Kubernetes Service (EKS) -* Helm: 3.2.0+ -* Elasticsearch, Kibana, APM Server: 6.8+, 7.1+, 8+ -* Enterprise Search: 7.7+, 8.x (Enterprise Search is not available in {{stack}} 9.0+) - -* Beats: 7.0+, 8+ -* Elastic Agent: 7.10+ (standalone), 7.14+ (Fleet), 8+ -* Elastic Maps Server: 7.11+, 8+ -* Logstash: 8.7+ - -ECK should work with all conformant installers as listed in these [FAQs](https://github.com/cncf/k8s-conformance/blob/master/faq.md#what-is-a-distribution-hosted-platform-and-an-installer). Distributions include source patches and so may not work as-is with ECK. - -Alpha, beta, and stable API versions follow the same [conventions used by Kubernetes](https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-versioning). - -Elastic Stack application images for the OpenShift-certified Elasticsearch (ECK) Operator are only available from version 7.10 and later. - -Check the full [Elastic support matrix](https://www.elastic.co/support/matrix#matrix_kubernetes) for more information. - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_installation.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_installation.md deleted file mode 100644 index 36ed69985f..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_installation.md +++ /dev/null @@ -1,33 +0,0 @@ -# Installation [k8s_installation] - -## FIPS compliant installation using Helm [k8s_fips_compliant_installation_using_helm] - -Set `image.fips=true` to install a FIPS-enabled version of the ECK Operator. Refer to [Install ECK using the Helm chart](../../../deploy-manage/deploy/cloud-on-k8s/install-using-helm-chart.md) for full Helm installation instructions. - -```sh -helm install elastic-operator elastic/eck-operator \ - -n elastic-system --create-namespace \ - --set=image.fips=true -``` - - -## FIPS compliant installation using manifests [k8s_fips_compliant_installation_using_manifests] - -The `StatefulSet` definition within the yaml installation manifest will need to be patched prior to installation to append `-fips` to the `spec.template.spec.containers[*].image` to install a FIPS-enabled version of the ECK Operator. Refer to [Install ECK using the YAML manifests](../../../deploy-manage/deploy/cloud-on-k8s/install-using-yaml-manifest-quickstart.md) for full manifest installation instructions. - -::::{note} -`${ECK_VERSION}` in the following command needs to be replaced with the version of the Operator that is to be installed. -:::: - - -```sh -curl -s https://download.elastic.co/downloads/eck/${ECK_VERSION}/operator.yaml | sed -r 's#(image:.*eck-operator)(:.*)#\1-fips\2#' | kubectl apply -f - -``` - -If the Operator has already been installed using the manifests, the installation can be patched instead: - -```sh -kubectl patch sts elastic-operator -n elastic-system -p '{"spec":{"template":{"spec":{"containers":[{"name":"manager", "image":"docker.elastic.co/eck/eck-operator-fips:${ECK_VERSION}"}]}}}}' -``` - - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md deleted file mode 100644 index 0501980ca6..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md +++ /dev/null @@ -1,17 +0,0 @@ -# Learn more about ECK [k8s_learn_more_about_eck] - -* [Orchestrate Elasticsearch on Kubernetes](https://www.elastic.co/elasticsearch-kubernetes) -* [ECK post on the Elastic Blog](https://www.elastic.co/blog/introducing-elastic-cloud-on-kubernetes-the-elasticsearch-operator-and-beyond?elektra=products&storm=sub1) -* [Getting Started With Elastic Cloud on Kubernetes (ECK)](https://www.youtube.com/watch?v=PIJmlYBIFXM) -* [Running the Elastic Stack on Kubernetes with ECK](https://www.youtube.com/watch?v=Wf6E3vkvEFM) - - -## Ask for help [k8s-ask-for-help] - -If you are an existing Elastic customer with an active support contract, you can create a case in the [Elastic Support Portal](https://support.elastic.co/). Kindly attach an [ECK diagnostic](../../../troubleshoot/deployments/cloud-on-k8s/run-eck-diagnostics.md) when opening your case. - -Alternatively, or if you do not have a support contract, and if you are unable to find a solution to your problem with the information provided in these documents, ask for help: - -* [ECK Discuss forums](https://discuss.elastic.co/c/eck) to ask any question -* [Github issues](https://github.com/elastic/cloud-on-k8s/issues) for bugs and feature requests - diff --git a/raw-migrated-files/toc.yml b/raw-migrated-files/toc.yml index 32a13af50d..774cbc1246 100644 --- a/raw-migrated-files/toc.yml +++ b/raw-migrated-files/toc.yml @@ -14,26 +14,13 @@ toc: - file: cloud-on-k8s/cloud-on-k8s/index.md children: - file: cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md - - file: cloud-on-k8s/cloud-on-k8s/k8s_installation.md - - file: cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-air-gapped.md - file: cloud-on-k8s/cloud-on-k8s/k8s-custom-http-certificate.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-deploy-eck.md - file: cloud-on-k8s/cloud-on-k8s/k8s-es-secure-settings.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-fips.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-install-yaml-manifests.md - file: cloud-on-k8s/cloud-on-k8s/k8s-installing-eck.md - file: cloud-on-k8s/cloud-on-k8s/k8s-orchestration.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-overview.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md - file: cloud-on-k8s/cloud-on-k8s/k8s-rotate-credentials.md - file: cloud-on-k8s/cloud-on-k8s/k8s-saml-authentication.md - file: cloud-on-k8s/cloud-on-k8s/k8s-securing-stack.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-security.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-services.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-supported.md - file: cloud-on-k8s/cloud-on-k8s/k8s-tls-certificates.md - file: cloud-on-k8s/cloud-on-k8s/k8s-upgrading-stack.md - file: cloud-on-k8s/cloud-on-k8s/k8s-users-and-roles.md