Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
d5a8901
some stuff
shainaraskas Mar 3, 2025
334c967
more
shainaraskas Mar 4, 2025
47056c7
more
shainaraskas Mar 4, 2025
d370ff1
more"
shainaraskas Mar 5, 2025
c79b7f5
more subs
shainaraskas Mar 5, 2025
9aaa01a
so much stuff
shainaraskas Mar 5, 2025
25a613e
more
shainaraskas Mar 5, 2025
513a428
Merge branch 'main' into self-managed
shainaraskas Mar 5, 2025
3127d42
errors batch
shainaraskas Mar 5, 2025
b8653f9
more errors
shainaraskas Mar 6, 2025
a182f40
kib
shainaraskas Mar 7, 2025
90e854a
air gapped
shainaraskas Mar 7, 2025
a36e79e
stuff
shainaraskas Mar 7, 2025
5ca3a87
Merge branch 'main' into self-managed
shainaraskas Mar 7, 2025
4a5b52d
conflicts
shainaraskas Mar 7, 2025
aba29a1
overview
shainaraskas Mar 7, 2025
4fe35e9
getting closer
shainaraskas Mar 7, 2025
82d1037
more
shainaraskas Mar 10, 2025
3fb382a
Merge branch 'main' into self-managed
shainaraskas Mar 10, 2025
ace2bcf
clean up some labels
shainaraskas Mar 10, 2025
35ec379
Merge branch 'self-managed' of github.com:elastic/docs-content into s…
shainaraskas Mar 10, 2025
46a4cbb
fix more annotations
shainaraskas Mar 10, 2025
baa6aa1
broken window
shainaraskas Mar 10, 2025
4bf536f
Merge branch 'main' into self-managed
shainaraskas Mar 10, 2025
e5fb206
toc cleanup
shainaraskas Mar 10, 2025
9c16fd2
more cleanup
shainaraskas Mar 10, 2025
3c0e693
next steps
shainaraskas Mar 10, 2025
a8d5b23
tools + apis
shainaraskas Mar 10, 2025
f410023
little fix
shainaraskas Mar 10, 2025
9f06678
fewer redirects
shainaraskas Mar 10, 2025
fa86a58
Merge branch 'main' into self-managed
shainaraskas Mar 10, 2025
9840696
fix errors
shainaraskas Mar 10, 2025
95d5fa5
Merge branch 'self-managed' of github.com:elastic/docs-content into s…
shainaraskas Mar 10, 2025
cecb26a
use snippet
shainaraskas Mar 10, 2025
36d2f93
fix links"
shainaraskas Mar 10, 2025
b245873
more fixes
shainaraskas Mar 10, 2025
9d18e87
more fix
shainaraskas Mar 10, 2025
f8aadbe
more
shainaraskas Mar 10, 2025
3ccb9f7
more fixes
shainaraskas Mar 10, 2025
cd024e9
syntax fix
shainaraskas Mar 10, 2025
a47ae9f
Merge branch 'main' into self-managed
shainaraskas Mar 11, 2025
95cc162
Update deploy-manage/deploy/elastic-cloud/restrictions-known-problems.md
shainaraskas Mar 11, 2025
1a5e88d
too many hashes
shainaraskas Mar 11, 2025
09481d9
Merge branch 'self-managed' of github.com:elastic/docs-content into s…
shainaraskas Mar 11, 2025
db2a35b
Merge branch 'main' into self-managed
shainaraskas Mar 13, 2025
089e652
Apply suggestions from code review
shainaraskas Mar 13, 2025
9ef75ba
add kibana command line tools
shainaraskas Mar 13, 2025
80b8390
changes related to subs=true
shainaraskas Mar 13, 2025
406a737
tidy
shainaraskas Mar 13, 2025
a6cba55
Merge branch 'main' into self-managed
shainaraskas Mar 13, 2025
8ad2631
additional connection
shainaraskas Mar 13, 2025
94acbc4
Merge branch 'self-managed' of github.com:elastic/docs-content into s…
shainaraskas Mar 13, 2025
a2c3c24
thanks colleen
shainaraskas Mar 13, 2025
3ccfdf3
better start-stop page"
shainaraskas Mar 13, 2025
9ab2541
fix anchor
shainaraskas Mar 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,10 @@ To change your subscription level:

### Feature usage notifications [ec_feature_usage_notifications]

:::{applies_to}
:hosted: all
:::
```{applies_to}
deployment:
ess: all
```

If you try to change your subscription to a lower level, but you are using features that belong either to your current level or to a higher one, you need to make some changes before you can proceed, as described in **Review required feature changes**.

Expand Down
10 changes: 10 additions & 0 deletions deploy-manage/deploy/_snippets/installation-order.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
If you're deploying the {{stack}} in a self-managed cluster, then install the {{stack}} products you want to use in the following order:

* {{es}}
* {{kib}}
* [Logstash](logstash://reference/index.md)
* [{{agent}}](/reference/ingestion-tools/fleet/index.md) or [Beats](asciidocalypse://docs/beats/docs/reference/index.md)
* [APM](/solutions/observability/apps/application-performance-monitoring-apm.md)
* [Elasticsearch Hadoop](elasticsearch-hadoop://reference/index.md)

Installing in this order ensures that the components each product depends on are in place.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
When installing the {{stack}}, you must use the same version across the entire stack. For example, if you are using {{es}} {{stack-version}}, you install Beats {{stack-version}}, APM Server {{stack-version}}, Elasticsearch Hadoop {{stack-version}}, {{kib}} {{stack-version}}, and Logstash {{stack-version}}.

If you’re upgrading an existing installation, see [](/deploy-manage/upgrade.md) for information about how to ensure compatibility with {{stack-version}}.
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,7 @@ You can specify `--log-opt max-size` and `--log-opt max-file` to define the Dock

1. Update `/etc/systemd/system/docker.service.d/docker.conf`. If the file path and file do not exist, create them first.

```sh
```ini
[Unit]
Description=Docker Service
After=multi-user.target
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ If your custom images follow the naming convention adopted by the official image

For more information, check the following references:

* [Elasticsearch documentation on Using custom Docker images](/deploy-manage/deploy/self-managed/install-elasticsearch-with-docker.md#_c_customized_image)
* [Elasticsearch documentation on Using custom Docker images](/deploy-manage/deploy/self-managed/install-elasticsearch-docker-configure.md#_c_customized_image)
* [Google Container Registry](https://cloud.google.com/container-registry/docs/how-to)
* [Azure Container Registry](https://docs.microsoft.com/en-us/azure/container-registry/)
* [Amazon Elastic Container Registry](https://docs.aws.amazon.com/AmazonECR/latest/userguide/what-is-ecr.html)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ To deploy the ECK operator:

1. Install Elastic's [custom resource definitions](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) with [`create`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_create/):

```sh
```sh subs=true
kubectl create -f https://download.elastic.co/downloads/eck/{{eck_version}}/crds.yaml
```

Expand All @@ -58,7 +58,7 @@ To deploy the ECK operator:

2. Using [`kubectl apply`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_apply/), install the operator with its RBAC rules:

```sh
```sh subs=true
kubectl apply -f https://download.elastic.co/downloads/eck/{{eck_version}}/operator.yaml
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This page shows the installation steps to deploy ECK in Openshift:

1. Apply the manifests the same way as described in [](./install-using-yaml-manifest-quickstart.md) document:

```shell
```shell subs=true
oc create -f https://download.elastic.co/downloads/eck/{{eck_version}}/crds.yaml
oc apply -f https://download.elastic.co/downloads/eck/{{eck_version}}/operator.yaml
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ The operator itself must be connected to the service mesh to deploy and manage E

2. Install ECK:

```sh
```sh subs=true
kubectl create -f https://download.elastic.co/downloads/eck/{{eck_version}}/crds.yaml
kubectl apply -f https://download.elastic.co/downloads/eck/{{eck_version}}/operator.yaml
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ These instructions have been tested with Linkerd 2.7.0.

In order to connect the operator to the service mesh, Linkerd sidecar must be injected into the ECK deployment. This can be done during installation as follows:

```sh
```sh subs=true
kubectl create -f https://download.elastic.co/downloads/eck/{{eck_version}}/crds.yaml
linkerd inject https://download.elastic.co/downloads/eck/{{eck_version}}/operator.yaml | kubectl apply -f -
```
Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/deploy/deployment-comparison.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

# Compare deployment options
# Detailed deployment comparison

This reference provides detailed comparisons of features and capabilities across Elastic's deployment options: self-managed deployments, Elastic Cloud Hosted, and Serverless. For a high-level overview of deployment types and guidance on choosing between them, see the [overview](../deploy.md).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Elasticsearch APIs
$$$ec-restrictions-apis-kibana$$$

Kibana APIs
: There are no rate limits restricting your use of the Kibana APIs. However, Kibana features are affected by the [Kibana configuration settings](../self-managed/configure.md), not all of which are supported in {{ecloud}}. For a list of what settings are currently supported, check [Add Kibana user settings](edit-stack-settings.md). For all details about using the Kibana APIs, check the [Kibana API reference documentation](https://www.elastic.co/guide/en/kibana/current/api.html).
: There are no rate limits restricting your use of the Kibana APIs. However, Kibana features are affected by the [Kibana configuration settings](kibana://reference/configuration-reference.md), not all of which are supported in {{ecloud}}. For a list of what settings are currently supported, check [Add Kibana user settings](edit-stack-settings.md). For all details about using the Kibana APIs, check the [Kibana API reference documentation](https://www.elastic.co/docs/api/doc/kibana/).


## Transport client [ec-restrictions-transport-client]
Expand Down
107 changes: 105 additions & 2 deletions deploy-manage/deploy/self-managed.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,111 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/elasticsearch/reference/current/dependencies-versions.html
applies_to:
deployment:
self:
---

# Self-managed cluster [dependencies-versions]
# Self-managed cluster

See [Elastic Stack Third-party Dependencices](https://artifacts.elastic.co/reports/dependencies/dependencies-current.html) for the complete list of dependencies for {{es}}.
If you want to install Elastic on your own premises without the assistance of an [orchestrator](/deploy-manage/deploy.md#about-orchestration), then you can deploy a self-managed cluster. If you deploy a self-managed cluster, then you have complete control and responsibility over every aspect of your Elastic deployment.

To quickly set up {{es}} and {{kib}} in Docker for local development or testing, jump to [](/deploy-manage/deploy/self-managed/local-development-installation-quickstart.md).

:::{admonition} Simplify the deployment process
Self-managed clusters are useful for local development, and for exploring Elastic features. However, Elastic offers several deployment options that can simplify the process of deploying and managing multi-node deployments, especially in production. They also allow you to deploy and manage multiple deployments from a single surface.

Managed by Elastic:
* [{{serverless-full}}](/deploy-manage/deploy/elastic-cloud/serverless.md)
* [{{ech}}](/deploy-manage/deploy/elastic-cloud/cloud-hosted.md)

Self-hosted options:
* [{{eck}}](/deploy-manage/deploy/cloud-on-k8s.md)
* [{{ece}}](/deploy-manage/deploy/cloud-enterprise.md)

For a comparison of these deployment options, refer to [Choosing your deployment type](/deploy-manage/deploy.md#choosing-your-deployment-type) and [](/deploy-manage/deploy/deployment-comparison.md).
:::

## Section overview

This section focuses on deploying {{es}} and {{kib}} without an orchestrator.

Depending on your use case, you might need to deploy other components, such as APM, Fleet, or Logstash. Deploying those components is not covered in this section. [Learn more about optional components](/get-started/the-stack.md).

This section covers the following tasks:

### Deploying Elasticsearch

Learn how to install and configure {{es}}. {{es}} is the distributed search and analytics engine, scalable data store, and vector database at the heart of all Elastic solutions.

* [](/deploy-manage/deploy/self-managed/installing-elasticsearch.md)
* [](/deploy-manage/deploy/self-managed/important-system-configuration.md): Prepare your environment for an {{es}} installation.
* [](/deploy-manage/deploy/self-managed/installing-elasticsearch.md#installation-methods): Install and run {{es}} using one of our install packages or container images.
* [](/deploy-manage/deploy/self-managed/local-development-installation-quickstart.md): Quickly set up {{es}} and {{kib}} in Docker for local development or testing.
* [](/deploy-manage/deploy/self-managed/configure-elasticsearch.md): Learn how to make configuration changes to {{es}}
* [](/deploy-manage/deploy/self-managed/important-settings-configuration.md): Learn about key settings required for production environments.
* [](/deploy-manage/deploy/self-managed/plugins.md): Learn about how to extend {{es}} functionality with plugins.

:::{note}
For a complete list of settings that you can apply to your {{es}} cluster, refer to the [Elasticsearch configuration reference](elasticsearch://reference/elasticsearch/configuration-reference/index.md).
:::

### Deploying Kibana

After you deploy {{es}}, you can install {{kib}}. {{kib}} provides the user interface for all Elastic solutions. It’s a powerful tool for [visualizing and analyzing](/explore-analyze/index.md) your data, and for managing and monitoring the {{stack}}. Although {{kib}} is not required to use {{es}}, it's required for most [use cases](/solutions/index.md).

* [](/deploy-manage/deploy/self-managed/install-kibana.md): Install {{kib}} using one of our install packages or container images, and enroll {{kib}} with your {{es}} cluster.
* [](/deploy-manage/deploy/self-managed/configure-kibana.md): Learn how to make configuration changes to {{kib}}.
* [](/deploy-manage/deploy/self-managed/access-kibana.md): Learn how to access {{kib}} using a web browser.

### Installing in air gapped environments

Some components of the {{stack}} require additional configuration and local dependencies in order to deploy in environments without internet access.

Refer to [](/deploy-manage/deploy/self-managed/air-gapped-install.md) to learn how to install {{es}}, {{kib}}, and optional components in an environment without internet access.

### Tools and APIs

Review a list of all of the resources that you can use to interact with your self-managed cluster, including tools, APIs, client libraries, and more.

[](/deploy-manage/deploy/self-managed/tools-apis.md).

## Other important sections

Review these other sections for critical information about securing and managing your self-managed cluster.

### Secure and control access

Learn how to secure your Elastic environment to restrict access to only authorized parties, and allow communication between your environment and external parties.

* [](/deploy-manage/security.md): Learn about security features that prevent bad actors from tampering with your data, and encrypt communications to, from, and within your cluster.
* [Users and roles](/deploy-manage/users-roles/cluster-or-deployment-auth.md): Set up authentication and authorization for your cluster, and learn about the underlying security technologies that {{es}} uses to authenticate and authorize requests internally and across services.
* [](/deploy-manage/manage-spaces.md): Learn how to organize content in {{kib}}, and restrict access to this content to specific users.
* [](/deploy-manage/api-keys.md): Authenticate and authorize programmatic access to your deployments and {{es}} resources.
* [](/deploy-manage/manage-connectors.md): Manage connection information between Elastic and third-party systems.
* [](/deploy-manage/remote-clusters/remote-clusters-self-managed.md): Enable communication between {{es}} clusters to support [cross-cluster replication](/deploy-manage/tools/cross-cluster-replication.md) and [cross-cluster search](/solutions/search/cross-cluster-search.md).

### Administer and maintain

Monitor the performance of your Elastic environment, administer your license, set up backup and resilience tools, and maintain the health of your environment.

* [](/deploy-manage/tools.md): Learn about the tools available to safeguard data, ensure continuous availability, and maintain resilience in your {{es}} environment.
* [](/deploy-manage/monitor.md): View health and performance data for Elastic components, and receive recommendations and insights.
* [](/deploy-manage/license.md): Learn how to manage your Elastic license.
* [](/deploy-manage/maintenance/start-stop-services.md): Learn how to isolate or deactivate parts of your Elastic environment to perform maintenance, or restart parts of Elastic.
* [](/deploy-manage/maintenance/add-and-remove-elasticsearch-nodes.md): Learn how to add nodes to a cluster or remove them from a cluster to change the size and capacity of {{es}}.

### Upgrade

You can [upgrade your Elastic environment](/deploy-manage/upgrade.md) to gain access to the latest features.

### Design guidance

Learn how to design a production-ready Elastic environment.

* [](/deploy-manage/production-guidance.md): Review tips and guidance that you can use to design a production environment that matches your workloads, policies, and deployment needs.
* [](/deploy-manage/reference-architectures.md): Explore blueprints for deploying clusters tailored to different use cases.

### Architectural information

In the [](/deploy-manage/distributed-architecture.md) section, learn about the architecture of {{es}} and {{kib}}, and how Elastic stores and retrieves data and executes tasks in clusters with multiple nodes.
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
When you start {{es}} for the first time, the following security configuration occurs automatically:

* [Certificates and keys](/deploy-manage/security/security-certificates-keys.md#stack-security-certificates) for TLS are generated for the transport and HTTP layers.
* The TLS configuration settings are written to `elasticsearch.yml`.
* A password is generated for the `elastic` user.
* An enrollment token is generated for {{kib}}, which is valid for 30 minutes.

You can then start {{kib}} and enter the enrollment token. This token automatically applies the security settings from your {{es}} cluster, authenticates to {{es}} with the built-in `kibana` service account, and writes the security configuration to `kibana.yml`.

::::{note}
There are [some cases](/deploy-manage/security/security-certificates-keys.md#stack-skip-auto-configuration) where security can’t be configured automatically because the node startup process detects that the node is already part of a cluster, or that security is already configured or explicitly disabled.
::::
7 changes: 7 additions & 0 deletions deploy-manage/deploy/self-managed/_snippets/ca-cert.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
If your library doesn’t support a method of validating the fingerprint, the auto-generated CA certificate is created in the following directory on each {{es}} node:

```sh subs=true
{{es-conf}}{{slash}}certs{{slash}}http_ca.crt
```

Copy the `http_ca.crt` file to your machine and configure your client to use this certificate to establish trust when it connects to {{es}}.
14 changes: 14 additions & 0 deletions deploy-manage/deploy/self-managed/_snippets/ca-fingerprint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Copy the fingerprint value that’s output to your terminal when {{es}} starts, and configure your client to use this fingerprint to establish trust when it connects to {{es}}.

If the auto-configuration process already completed, you can still obtain the fingerprint of the security certificate by running the following command. The path is to the auto-generated CA certificate for the HTTP layer.

```sh
openssl x509 -fingerprint -sha256 -in config/certs/http_ca.crt
```

The command returns the security certificate, including the fingerprint. The `issuer` should be `{{es}} security auto-configuration HTTP CA`.

```sh
issuer= /CN=Elasticsearch security auto-configuration HTTP CA
SHA256 Fingerprint=<fingerprint>
```
32 changes: 32 additions & 0 deletions deploy-manage/deploy/self-managed/_snippets/check-es-running.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
You can test that your {{es}} node is running by sending an HTTPS request to port `9200` on `localhost`:

```sh subs=true
curl --cacert {{es-conf}}{{slash}}certs{{slash}}http_ca.crt {{escape}} <1>
-u elastic:$ELASTIC_PASSWORD https://localhost:9200 <2>
```
1. `--cacert`: Path to the generated `http_ca.crt` certificate for the HTTP layer.
2. Ensure that you use `https` in your call, or the request will fail.



The call returns a response like this:

```js
{
"name" : "Cp8oag6",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "AT69_T_DTp-1qgIJlatQqA",
"version" : {
"number" : "9.0.0-SNAPSHOT",
"build_type" : "tar",
"build_hash" : "f27399d",
"build_flavor" : "default",
"build_date" : "2016-03-30T09:51:41.449Z",
"build_snapshot" : false,
"lucene_version" : "10.0.0",
"minimum_wire_compatibility_version" : "1.2.3",
"minimum_index_compatibility_version" : "1.2.3"
},
"tagline" : "You Know, for Search"
}
```
21 changes: 21 additions & 0 deletions deploy-manage/deploy/self-managed/_snippets/cmd-line-config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{{es}} loads its configuration from the following location by default:

```sh subs=true
{{es-conf}}{{slash}}elasticsearch.yml
```

The format of this config file is explained in [](/deploy-manage/deploy/self-managed/configure-elasticsearch.md).

Any settings that can be specified in the config file can also be specified on the command line, using the `-E` syntax as follows:

```sh subs=true
.{{slash}}bin{{slash}}elasticsearch{{auto}} -Ecluster.name=my_cluster -Enode.name=node_1
```

:::{note}
Values that contain spaces must be surrounded with quotes. For instance `-Epath.logs="C:\My Logs\logs"`.
:::

:::{tip}
Typically, any cluster-wide settings (like `cluster.name`) should be added to the `elasticsearch.yml` config file, while any node-specific settings such as `node.name` could be specified on the command line.
::::
12 changes: 12 additions & 0 deletions deploy-manage/deploy/self-managed/_snippets/connect-clients.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
% This file is reused in each of the installation pages. Ensure that any changes
% you make to this file are applicable across all installation environments.

When you start {{es}} for the first time, TLS is configured automatically for the HTTP layer. A CA certificate is generated and stored on disk at:

```sh subs=true
{{es-conf}}{{slash}}certs{{slash}}http_ca.crt
```

The hex-encoded SHA-256 fingerprint of this certificate is also output to the terminal. Any clients that connect to {{es}}, such as the [{{es}} Clients](https://www.elastic.co/guide/en/elasticsearch/client/index.html), {{beats}}, standalone {{agent}}s, and {{ls}} must validate that they trust the certificate that {{es}} uses for HTTPS. {{fleet-server}} and {{fleet}}-managed {{agent}}s are automatically configured to trust the CA certificate. Other clients can establish trust by using either the fingerprint of the CA certificate or the CA certificate itself.

If the auto-configuration process already completed, you can still obtain the fingerprint of the security certificate. You can also copy the CA certificate to your machine and configure your client to use it.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
In production, we recommend you run {{es}} on a dedicated host or as a primary service. Several {{es}} features, such as automatic JVM heap sizing, assume that {{es}} is the only resource-intensive application on the host or container. For example, you might run {{metricbeat}} alongside {{es}} for cluster statistics, but a resource-heavy {{ls}} deployment should be on its own host.
Loading