Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
a4472c7
merging shaina's PR with my WIP changes
eedugon Mar 26, 2025
8f374d0
security certificates work in progress
eedugon Mar 26, 2025
0fa5251
HTTPS setup ongoing
eedugon Mar 26, 2025
e4601e1
kibana https updated
eedugon Mar 26, 2025
155e78f
syncing tabs
eedugon Mar 26, 2025
f6da69c
working in progress
eedugon Mar 26, 2025
613a131
automatic and manual configs for self-managed almost completed
eedugon Mar 27, 2025
602e280
basic and basic plus concepts refined
eedugon Mar 27, 2025
4dd4b33
TLS certificates extra landing pages added
eedugon Mar 27, 2025
6185393
redirect for self-setup.md
eedugon Mar 27, 2025
0a15585
automatic security setup file renamed to self-auto-setup.md
eedugon Mar 27, 2025
6e86fa6
final refinements before ready for review
eedugon Mar 28, 2025
f0626ec
Merge remote-tracking branch 'origin/main' into security_comms
eedugon Mar 28, 2025
36b15c7
moved tutorial
eedugon Mar 28, 2025
d3ba8d5
fixing some links
eedugon Mar 28, 2025
d820457
Apply suggestions from code review
eedugon Mar 29, 2025
fc987ad
deploy guides updated
eedugon Mar 31, 2025
ea7867b
changes per review comments
eedugon Mar 31, 2025
7a7241d
removing badge in section
eedugon Mar 31, 2025
c4fa413
fixing various links
eedugon Mar 31, 2025
d2b767a
Merge remote-tracking branch 'origin/main' into security_comms
eedugon Mar 31, 2025
44ed1bf
kibana security doc reduced. finding home
eedugon Mar 31, 2025
8183345
comparison table updated
eedugon Mar 31, 2025
bd3cb74
getting closer
eedugon Mar 31, 2025
87f634b
Update deploy-manage/security/self-setup.md
eedugon Apr 1, 2025
6833ffc
refinement completed
eedugon Apr 1, 2025
e634418
merging main without redirects
eedugon Apr 1, 2025
1fe942a
new redirects.yml added
eedugon Apr 1, 2025
3e9cf0a
Merge branch 'main' into security_comms
shainaraskas Apr 1, 2025
f82bfbd
Apply suggestions from code review
eedugon Apr 1, 2025
d0b5df0
review in progress
eedugon Apr 1, 2025
2703f92
Apply suggestions from code review
eedugon Apr 1, 2025
23a9ae2
Update deploy-manage/security/_snippets/cluster-comparison.md
eedugon Apr 1, 2025
7da7542
Update deploy-manage/security/_snippets/cluster-comparison.md
eedugon Apr 1, 2025
beca853
working on reviews
eedugon Apr 1, 2025
8da916c
working on reviews
eedugon Apr 1, 2025
091d773
working on reviews
eedugon Apr 1, 2025
8c87940
working on reviews
eedugon Apr 1, 2025
b9c7373
working on reviews
eedugon Apr 1, 2025
9108fb2
working on reviews
eedugon Apr 1, 2025
3b7e16f
applies_to and substitutions fixed
eedugon Apr 1, 2025
8044f47
Update deploy-manage/security/set-up-basic-security-plus-https.md
eedugon Apr 1, 2025
6bacccb
Update deploy-manage/security/set-up-basic-security-plus-https.md
eedugon Apr 1, 2025
5da9e04
Merge branch 'main' into security_comms
eedugon Apr 1, 2025
dc296f1
install stack demo secure moved to raw content
eedugon Apr 2, 2025
9a69bec
removed empty line
eedugon Apr 2, 2025
db53e13
Merge branch 'main' into security_comms
eedugon Apr 2, 2025
4a31e1d
final refinement
eedugon Apr 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion deploy-manage/deploy/cloud-on-k8s/accessing-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ This section explains how to access and customize the Kubernetes services and se

For advanced use cases related to exposing and accessing orchestrated applications, see:

* [](/deploy-manage/security/secure-http-communications.md): Configuration options for the HTTP SSL certificates, including integration with certificate management systems such as [cert-manager](https://cert-manager.io/).
* [](/deploy-manage/security/secure-cluster-communications.md): Configuration options for the HTTP SSL certificates, including integration with certificate management systems such as [cert-manager](https://cert-manager.io/).
* [](./service-meshes.md): Connect ECK and your managed deployments to service mesh implementations such as [Istio](https://istio.io) and [Linkerd](https://linkerd.io).
* [](./requests-routing-to-elasticsearch-nodes.md): Create custom services to expose different node types.
* [Use Ingress to expose {{es}} or {{kib}}](./managing-deployments-using-helm-chart.md#k8s-eck-stack-ingress): Helm based installation also facilitates the creation of Ingress resources.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,6 @@ Now that you know how to use the APM keystore and customize the server configura

By default the operator manages a private CA and generates a self-signed certificate used to secure the communication between APM agents and the server.

This behavior and the relevant configuration is identical to what is done for Elasticsearch and Kibana. Check [Setting up your own certificate](/deploy-manage/security/secure-http-communications.md) for more information on how to use your own certificate to configure the TLS endpoint of the APM Server.
This behavior and the relevant configuration is identical to what is done for Elasticsearch and Kibana. Check [Setting up your own certificate](/deploy-manage/security/secure-cluster-communications.md) for more information on how to use your own certificate to configure the TLS endpoint of the APM Server.

For more details on how to configure the APM agents to work with custom certificates, check the [APM agents documentation](https://www.elastic.co/guide/en/apm/agent/index.html).
2 changes: 1 addition & 1 deletion deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ To deploy {{agent}} in clusters with the Pod Security Policy admission controlle

## Customize {{fleet-server}} Service [k8s-elastic-agent-fleet-configuration-customize-fleet-server-service]

By default, ECK creates a Service for {{fleet-server}} that {{agents}} can connect through. You can customize it using the `http` configuration element. Check more information on how to [make changes](accessing-services.md) to the Service and [customize](/deploy-manage/security/secure-http-communications.md) the TLS configuration.
By default, ECK creates a Service for {{fleet-server}} that {{agents}} can connect through. You can customize it using the `http` configuration element. Check more information on how to [make changes](accessing-services.md) to the Service and [customize](/deploy-manage/security/secure-cluster-communications.md) the TLS configuration.


## Control {{fleet}} policy selection [k8s-elastic-agent-control-fleet-policy-selection]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Before deploying and running ECK in production, review the basic and advanced se

## TLS/SSL Certificates

* [Secure HTTP communications](/deploy-manage/security/secure-http-communications.md): Customize the service and TLS certificates used for transport traffic.
* [Secure HTTP communications](/deploy-manage/security/secure-cluster-communications.md): Customize the service and TLS certificates used for transport traffic.
* [Transport settings](../../security/k8s-transport-settings.md): Customize the service and TLS certificates used for transport traffic.

## Traffic handling
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ In order to make requests to the [{{es}} API](elasticsearch://reference/elastics
PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')
```

2. Request the [{{es}} root API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-info). You can do so from inside the Kubernetes cluster or from your local workstation. For demonstration purposes, certificate verification is disabled using the `-k` curl flag; however, this is not recommended outside of testing purposes. Refer to [Setup your own certificate](/deploy-manage/security/secure-http-communications.md#k8s-setting-up-your-own-certificate) for more information.
2. Request the [{{es}} root API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-info). You can do so from inside the Kubernetes cluster or from your local workstation. For demonstration purposes, certificate verification is disabled using the `-k` curl flag; however, this is not recommended outside of testing purposes. Refer to [Setup your own certificate](/deploy-manage/security/secure-cluster-communications.md#k8s-setting-up-your-own-certificate) for more information.

* From inside the Kubernetes cluster:

Expand Down
6 changes: 3 additions & 3 deletions deploy-manage/deploy/cloud-on-k8s/http-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,17 @@ This functionality is in technical preview and may be changed or removed in a fu

By default a `ClusterIP` [service](https://kubernetes.io/docs/concepts/services-networking/service/) is created and associated with the Elastic Maps Server deployment. If you want to expose maps externally with a [load balancer](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer), it is recommended to include a custom DNS name or IP in the self-generated certificate.

Refer to [Reserve static IP and custom domain](/deploy-manage/security/secure-http-communications.md#k8s-static-ip-custom-domain) for more details.
Refer to [Reserve static IP and custom domain](/deploy-manage/security/secure-cluster-communications.md#k8s-static-ip-custom-domain) for more details.


## Provide your own certificate [k8s-maps-http-custom-tls]

If you want to use your own certificate, the required configuration is identical to Elasticsearch. Check [Custom HTTP certificate](../../security/secure-http-communications.md).
If you want to use your own certificate, the required configuration is identical to Elasticsearch. Check [Custom HTTP certificate](../../security/secure-cluster-communications.md).


## Disable TLS [k8s-maps-http-disable-tls]

You can disable the generation of the self-signed certificate and hence disable TLS. Check [Disable TLS](/deploy-manage/security/secure-http-communications.md#k8s-disable-tls).
You can disable the generation of the self-signed certificate and hence disable TLS. Check [Disable TLS](/deploy-manage/security/secure-cluster-communications.md#k8s-disable-tls).

### Ingress and Kibana configuration [k8s-maps-ingress]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ If you already looked at the [Elasticsearch on ECK](elasticsearch-configuration.

* [Customize the Pod configuration](#k8s-kibana-pod-configuration)
* [Customize the product configuration](#k8s-kibana-configuration)
* [Manage HTTP settings](/deploy-manage/security/secure-http-communications.md#k8s-kibana-http-configuration)
* [Manage HTTP settings](/deploy-manage/security/secure-cluster-communications.md#k8s-kibana-http-configuration)
* [Use secure settings](../../security/k8s-secure-settings.md)
* [Install {{kib}} plugins](k8s-kibana-plugins.md)

Expand Down
8 changes: 4 additions & 4 deletions deploy-manage/deploy/cloud-on-k8s/kibana-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ The following sections describe how to customize a {{kib}} deployment to suit yo
* [Scaling out a {{kib}} deployment](k8s-kibana-advanced-configuration.md#k8s-kibana-scaling)

* [Secure settings](../../security/k8s-secure-settings.md#k8s-kibana-secure-settings)
* [HTTP Configuration](/deploy-manage/security/secure-http-communications.md#k8s-kibana-http-configuration)
* [HTTP Configuration](/deploy-manage/security/secure-cluster-communications.md#k8s-kibana-http-configuration)

* [Load balancer settings and TLS SANs](/deploy-manage/security/secure-http-communications.md#k8s-kibana-http-publish)
* [Provide your own certificate](/deploy-manage/security/secure-http-communications.md#k8s-kibana-http-custom-tls)
* [Disable TLS](/deploy-manage/security/secure-http-communications.md#k8s-kibana-http-disable-tls)
* [Load balancer settings and TLS SANs](/deploy-manage/security/secure-cluster-communications.md#k8s-kibana-http-publish)
* [Provide your own certificate](/deploy-manage/security/secure-cluster-communications.md#k8s-kibana-http-custom-tls)
* [Disable TLS](/deploy-manage/security/secure-cluster-communications.md#k8s-kibana-http-disable-tls)
* [Install {{kib}} plugins](k8s-kibana-plugins.md)

* [Autoscaling stateless applications](../../autoscaling/autoscaling-in-eck.md#k8s-stateless-autoscaling): Use [Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) for {{kib}} or other stateless applications.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ To deploy a simple [{{kib}}](/get-started/the-stack.md#stack-components-kibana)
kubectl port-forward service/quickstart-kb-http 5601
```

Open `https://localhost:5601` in your browser. Your browser will show a warning because the self-signed certificate configured by default is not verified by a known certificate authority and not trusted by your browser. You can temporarily acknowledge the warning for the purposes of this quick start but it is highly recommended that you [configure valid certificates](/deploy-manage/security/secure-http-communications.md#k8s-setting-up-your-own-certificate) for any production deployments.
Open `https://localhost:5601` in your browser. Your browser will show a warning because the self-signed certificate configured by default is not verified by a known certificate authority and not trusted by your browser. You can temporarily acknowledge the warning for the purposes of this quick start but it is highly recommended that you [configure valid certificates](/deploy-manage/security/secure-cluster-communications.md#k8s-setting-up-your-own-certificate) for any production deployments.

Login as the `elastic` user. The password can be obtained with the following command:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ spec:

## Provide your own certificate [k8s-logstash-http-custom-tls]

If you want to use your own certificate, the required configuration is similar to Elasticsearch. Configure the certificate in `api` Service. Check [Custom HTTP certificate](../../security/secure-http-communications.md).
If you want to use your own certificate, the required configuration is similar to Elasticsearch. Configure the certificate in `api` Service. Check [Custom HTTP certificate](../../security/secure-cluster-communications.md).

```yaml
apiVersion: logstash.k8s.elastic.co/v1alpha1
Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/deploy/self-managed/access-kibana.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Access {{kib}} through the web application on port 5601.
To remotely connect to {{kib}}, set [`server.host`](kibana://reference/configuration-reference/general-settings.md#server-host) to a non-loopback address.

:::{note}
For production deployments, you should always [secure {{kib}} with a certificate](/deploy-manage/security/secure-http-communications.md#encrypt-kibana-http) and access it over HTTPS.
For production deployments, you should always [secure {{kib}} with a certificate](/deploy-manage/security/secure-cluster-communications.md#encrypt-kibana-http) and access it over HTTPS.
:::

2. Log on to your account.
Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/remote-clusters/remote-clusters-cert.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsear

## Establish trust with a remote cluster [remote-clusters-security-cert]

To use {{ccr}} or {{ccs}} safely with remote clusters, enable security on all connected clusters and configure Transport Layer Security (TLS) on every node. Configuring TLS security on the transport interface is minimally required for remote clusters. For additional security, configure TLS on the [HTTP interface](../security/secure-http-communications.md) as well.
To use {{ccr}} or {{ccs}} safely with remote clusters, enable security on all connected clusters and configure Transport Layer Security (TLS) on every node. Configuring TLS security on the transport interface is minimally required for remote clusters. For additional security, configure TLS on the [HTTP interface](../security/secure-cluster-communications.md) as well.

All connected clusters must trust one another and be mutually authenticated with TLS on the transport interface. This means that the local cluster trusts the certificate authority (CA) of the remote cluster, and the remote cluster trusts the CA of the local cluster. When establishing a connection, all nodes will verify certificates from nodes on the other side. This mutual trust is required to securely connect a remote cluster, because all connected nodes effectively form a single security domain.

Expand Down
26 changes: 26 additions & 0 deletions deploy-manage/security/_snippets/kibana-client-https-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
When you ran the `elasticsearch-certutil` tool with the `http` option, it created a `/kibana` directory containing an `elasticsearch-ca.pem` file. You use this file to configure {{kib}} to trust the {{es}} CA for the HTTP layer.

1. Copy the `elasticsearch-ca.pem` file to the {{kib}} configuration directory, as defined by the `$KBN_PATH_CONF` path.
2. Open `kibana.yml` and add the following line to specify the location of the security certificate for the HTTP layer.

```yaml
elasticsearch.ssl.certificateAuthorities: $KBN_PATH_CONF/elasticsearch-ca.pem
```

3. Add the following line to specify the HTTPS URL for your {{es}} cluster.

```yaml
elasticsearch.hosts: https://<your_elasticsearch_host>:9200
```

4. Restart {{kib}}.

:::::{admonition} Connect to a secure monitoring cluster
If the Elastic monitoring features are enabled and you configured a separate {{es}} monitoring cluster, you can also configure {{kib}} to connect to the monitoring cluster through HTTPS. The steps are the same, but each setting is prefixed by `monitoring`. For example, `monitoring.ui.elasticsearch.hosts` and `monitoring.ui.elasticsearch.ssl.truststore.path`.

::::{note}
You must create a separate `elasticsearch-ca.pem` security file for the monitoring cluster.
::::

:::::

52 changes: 52 additions & 0 deletions deploy-manage/security/_snippets/kibana-https-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
You create a server certificate and private key for {{kib}}. {{kib}} uses this server certificate and corresponding private key when receiving connections from web browsers.

When you obtain a server certificate, you must set its subject alternative name (SAN) correctly to ensure that browsers will trust it. You can set one or more SANs to the {{kib}} server’s fully-qualified domain name (FQDN), hostname, or IP address. When choosing the SAN, pick whichever attribute you’ll use to connect to {{kib}} in your browser, which is likely the FQDN.

The following instructions create a Certificate Signing Request (CSR) for {{kib}}. A CSR contains information that a CA uses to generate and sign a security certificate. The certificate can be trusted (signed by a public, trusted CA) or untrusted (signed by an internal CA). A self-signed or internally-signed certificate is acceptable for development environments and building a proof of concept, but should not be used in a production environment.

::::{warning}
Before going to production, use a trusted CA such as [Let’s Encrypt](https://letsencrypt.org/) or your organization’s internal CA to sign the certificate. Using a signed certificate establishes browser trust for connections to {{kib}} for internal access or on the public internet.
::::


1. Generate a server certificate and private key for {{kib}}.

```shell
./bin/elasticsearch-certutil csr -name kibana-server -dns example.com,www.example.com
```

The CSR has a common name (CN) of `kibana-server`, a SAN of `example.com`, and another SAN of `www.example.com`.

This command generates a `csr-bundle.zip` file by default with the following contents:

```txt
/kibana-server
|_ kibana-server.csr
|_ kibana-server.key
```

2. Unzip the `csr-bundle.zip` file to obtain the `kibana-server.csr` unsigned security certificate and the `kibana-server.key` unencrypted private key.
3. Send the `kibana-server.csr` certificate signing request to your internal CA or trusted CA for signing to obtain a signed certificate. The signed file can be in different formats, such as a `.crt` file like `kibana-server.crt`.
4. Open `kibana.yml` and add the following lines to configure {{kib}} to access the server certificate and unencrypted private key.

```yaml
server.ssl.certificate: $KBN_PATH_CONF/kibana-server.crt
server.ssl.key: $KBN_PATH_CONF/kibana-server.key
```

::::{note}
`$KBN_PATH_CONF` contains the path for the {{kib}} configuration files. If you installed {{kib}} using archive distributions (`zip` or `tar.gz`), the path defaults to `$KBN_HOME/config`. If you used package distributions (Debian or RPM), the path defaults to `/etc/kibana`.
::::

5. Add the following line to `kibana.yml` to enable TLS for inbound connections.

```yaml
server.ssl.enabled: true
```

6. Start {{kib}}.

::::{note}
After making these changes, you must always access {{kib}} via HTTPS. For example, `https://<your_kibana_host>.com`.
::::

15 changes: 15 additions & 0 deletions deploy-manage/security/eck-tls.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
applies_to:
deployment:
eck: all
navigation_title: ECK
mapped_pages:
- https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-security.html
---

# Manage TLS certificates on ECK

All {{stack}} resources deployed by the ECK operator are secured by default. The operator sets up basic authentication and TLS to encrypt network traffic to, from, and within your {{es}} cluster and {{kib}} instances.

% TBD
(Pending to write a quick intro about use cases and links)
21 changes: 14 additions & 7 deletions deploy-manage/security/httprest-clients-security.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,17 @@ mapped_pages:
- https://www.elastic.co/guide/en/elasticsearch/reference/current/http-clients.html
---

# HTTP/REST clients and security [http-clients]
# Securing HTTP client applications

When connecting client applications to {{es}}, use these best practices:

- Always use HTTPS for all connections
- Validate server certificates to prevent man-in-the-middle attacks
- Use API keys or token-based authentication rather than basic auth where possible
- Implement appropriate connection pooling and retry mechanisms
- Consider mutual TLS for high-security environments

## HTTP/REST clients and security [http-clients]

The {{es}} {{security-features}} work with standard HTTP [basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) headers to authenticate users. Since {{es}} is stateless, this header must be sent with every request:

Expand All @@ -17,7 +27,7 @@ Authorization: Basic <TOKEN> <1>
Alternatively, you can use [token-based authentication services](../users-roles/cluster-or-deployment-auth/token-based-authentication-services.md).


## Client examples [http-clients-examples]
### Client examples [http-clients-examples]

This example uses `curl` without basic auth to create an index:

Expand All @@ -44,8 +54,7 @@ curl --user rdeniro:taxidriver -XPUT 'localhost:9200/idx'
}
```


## Secondary authorization [http-clients-secondary-authorization]
### Secondary authorization [http-clients-secondary-authorization]

Some APIs support secondary authorization headers for situations where you want tasks to run with a different set of credentials. For example, you can send the following header in addition to the basic authentication header:

Expand All @@ -64,9 +73,7 @@ es-secondary-authorization: ApiKey <TOKEN> <1>

1. The `<TOKEN>` is computed as `base64(API key ID:API key)`



## Client libraries over HTTP [http-clients-libraries]
### Client libraries over HTTP [http-clients-libraries]

For more information about using {{security-features}} with the language specific clients, refer to:

Expand Down
Loading
Loading