Skip to content

Commit 0bdb851

Browse files
committed
orchestrator stuff
1 parent 108dc41 commit 0bdb851

10 files changed

+384
-361
lines changed

deploy-manage/monitor.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,16 @@ deployment:
8181
eck:
8282
```
8383

84-
TODO
84+
In addition to monitoring your cluster or deployment health and performance, you need to monitor your orchestrator. Monitoring is especially important for orchestrators hosted on infrastructure that you control.
85+
86+
Learn how to enable monitoring of your orchestrator:
87+
88+
* [ECK operator metrics](/deploy-manage/monitor/orchestrators/eck-metrics-configuration.md): Open and secure a metrics endpoint that can be used to monitor the operator’s performance and health. This endpoint can be scraped by third-party Kubernetes monitoring tools.
89+
* [ECK platform monitoring](/deploy-manage/monitor/orchestrators/ece-platform-monitoring.md): Learn about how ECE collects monitoring data for your installation in the `logging-and-metrics` deployment, and how to access monitoring data.
90+
91+
:::{admonition} Monitoring {{ecloud}}
92+
Elastic monitors [{{ecloud}}](/deploy-manage/deploy/elastic-cloud.md) service metrics and performance as part of [our shared responsibility](https://www.elastic.co/cloud/shared-responsibility). We provide service availability information on our [service status page](/deploy-manage/cloud-organization/service-status.md).
93+
:::
8594

8695
## Logging
8796

deploy-manage/monitor/orchestrators.md

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,17 @@ applies_to:
77

88
# Monitoring orchestrators
99

10-
% What needs to be done: Write from scratch
10+
Your [orchestrator](/deploy-manage/deploy.md#about-orchestration) is an important part of your Elastic architecture. It automates the deployment and management of multiple Elastic clusters, handling tasks like scaling, upgrades, and monitoring. Like your cluster or deployment, you need to monitor your orchestrator to ensure that it is healthy and performant. Monitoring is especially important for orchestrators hosted on infrastructure that you control.
1111

12-
% GitHub issue: https://github.com/elastic/docs-projects/issues/350
12+
In this section, you'll learn how to enable monitoring of your orchestrator.
1313

14-
% Scope notes: Landing page to monitoring orchestrators (not deployments)
14+
* [ECK operator metrics](/deploy-manage/monitor/orchestrators/eck-metrics-configuration.md): Open and secure a metrics endpoint that can be used to monitor the operator’s performance and health. This endpoint can be scraped by third-party Kubernetes monitoring tools.
15+
* [ECK platform monitoring](/deploy-manage/monitor/orchestrators/ece-platform-monitoring.md): Learn about how ECE collects monitoring data for your installation in the `logging-and-metrics` deployment, and how to access monitoring data.
1516

16-
⚠️ **This page is a work in progress.** ⚠️
17+
:::{admonition} Monitoring {{ecloud}}
18+
Elastic monitors {{ecloud}} service metrics and performance as part of [our shared responsibility](https://www.elastic.co/cloud/shared-responsibility). We provide service availability information on our [service status page](/deploy-manage/cloud-organization/service-status.md).
19+
:::
20+
21+
:::{note}
22+
Orchestrator monitoring can sometimes augment cluster or deployment monitoring, but doesn't replace it. For information about monitoring your cluster or deployment, refer to [](/deploy-manage/monitor.md).
23+
:::

deploy-manage/monitor/orchestrators/ece-monitoring-ece-access.md

Lines changed: 24 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -6,52 +6,47 @@ applies_to:
66
ece: all
77
---
88

9-
# Access logs and metrics [ece-monitoring-ece-access]
9+
# Platform monitoring deployment logs and metrics [ece-monitoring-ece-access]
1010

11-
To access logs and metrics for your deployment:
12-
13-
1. [Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md).
14-
2. From the **Deployments** page, select your deployment.
15-
16-
Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
17-
18-
3. On the **Elasticsearch** page, the following logs and metrics are available:
19-
20-
Elasticsearch logs
21-
: Detailed logs related to cluster state
11+
On the **{{es}}** page, the following logs and metrics are available:
2212

23-
Elasticsearch metrics
24-
: Detailed metrics for CPU and memory usage, running processes, networking and file system performance, and more
13+
| Log or metric | Description |
14+
| --- | --- |
15+
| {{es}} logs | Detailed logs related to cluster state. |
16+
| {{es}} metrics | Detailed metrics for CPU and memory usage, running processes, networking and file system performance, and more. |
17+
| Proxy logs | Search and indexing requests that proxies have sent to the {{es}} cluster. |
2518

26-
Proxy logs
27-
: Search and indexing requests that proxies have sent to the Elasticsearch cluster
19+
If a {{kib}} instance exists for the deployment, the following {{kib}} logs and metrics are also available from the **{{kib}}** page:
2820

29-
If a Kibana instance exists for the deployment, the following Kibana logs and metrics are also available from the **Kibana** page:
21+
| Log or metric | Description |
22+
| --- | --- |
23+
| {{kib}} logs | Detailed logs related to instance state. |
24+
| {{kib}} metrics | Detailed metrics for CPU and memory usage, running processes, networking and file system performance, and more. |
25+
| Proxy logs | Requests that proxies have sent to the {{kib}} instance. |
3026

31-
Kibana logs
32-
: Detailed logs related to instance state
27+
## Access logs and metrics
3328

34-
Kibana metrics
35-
: Detailed metrics for CPU and memory usage, running processes, networking and file system performance, and more
29+
To access logs and metrics for your deployment:
3630

37-
Proxy logs
38-
: Requests that proxies have sent to the Kibana instance
31+
1. [Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md).
32+
2. From the **Deployments** page, select your deployment.
3933

34+
Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
4035

41-
::::{tip}
42-
If you have a license from 2018 or earlier, you might receive a warning that your cluster license is about to expire. Don’t panic, it isn’t really. Elastic Cloud Enterprise manages the cluster licenses so that you don’t have to. In rare cases, such as when a cluster is overloaded, it can take longer for Elastic Cloud Enterprise to reapply the cluster license. If you have a license from 2019 and later, you’ll receive a warning only when your full platform license is about to expire, which you’ll need to renew.
43-
::::
36+
3. Depending on the logs or metrics that you want to access, click on the **Elasticsearch** or **Kibana** page.
4437

38+
::::{tip}
39+
If you have a license from 2018 or earlier, you might receive a warning that your cluster license is about to expire. Don’t panic, it isn’t really. Elastic Cloud Enterprise manages the cluster licenses so that you don’t have to. In rare cases, such as when a cluster is overloaded, it can take longer for Elastic Cloud Enterprise to reapply the cluster license. If you have a license from 2019 and later, you’ll receive a warning only when your full platform license is about to expire, which you’ll need to renew.
40+
::::
4541

46-
1. Select one of the links and log in with the `elastic` user. If you do not know the password, you can [reset it first](../../users-roles/cluster-or-deployment-auth/built-in-users.md).
42+
4. Select one of the links and log in with the `elastic` user. If you do not know the password, you can [reset it first](../../users-roles/cluster-or-deployment-auth/built-in-users.md).
4743

4844
::::{tip}
4945
The password you specify must be for the `elastic` user on the `logging-and-metrics` cluster, where the logging and metrics indices are collected. If you need to reset the password for the user, make sure you reset for the `logging-and-metrics` cluster.
5046
::::
5147

5248

53-
After you select one of the links, Kibana opens and shows you a view of the monitoring metrics for the logs or metrics that you selected.
49+
After you select one of the links, {{kib}} opens and shows you a view of the monitoring metrics for the logs or metrics that you selected.
5450

5551

5652
If you are looking for an {{es}} or {{kib}} diagnostic to share with Elastic support, go to the **Operations** page for the deployment and download the diagnostic bundle to attach to your ticket. If logs or an ECE diagnostic are requested by Elastic support, please [run the ECE diagnostics tool](../../../troubleshoot/deployments/cloud-enterprise/run-ece-diagnostics-tool.md).
57-

deploy-manage/monitor/orchestrators/ece-monitoring-ece-set-retention.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ You might need to adjust the retention period for one of the following reasons:
1818
To customize the retention period, set up a custom lifecycle policy for logs and metrics indices:
1919

2020
1. [Create a new index lifecycle management (ILM) policy](../../../manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md) in the logging and metrics cluster.
21-
2. Create a new, legacy-style, index template that matches the data view (formerly *index pattern*) that you wish to customize lifecycle for.
21+
2. Create a new, legacy-style, index template that matches the data view (formerly *index pattern*) that you want to customize the lifecycle for.
2222
3. Specify a lifecycle policy in the index template settings.
2323
4. Choose a higher `order` for the template so the specified lifecycle policy will be used instead of the default.
2424

deploy-manage/monitor/orchestrators/ece-platform-monitoring.md

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -8,22 +8,25 @@ applies_to:
88

99
# ECE platform monitoring [ece-monitoring-ece]
1010

11-
Elastic Cloud Enterprise by default collects monitoring data for your installation using Filebeat and Metricbeat. This data gets sent as monitoring indices to a dedicated `logging-and-metrics` deployment that is created whenever you install Elastic Cloud Enterprise on your first host. Data is collected on every host that is part of your Elastic Cloud Enterprise installation and includes:
11+
By default, {{ece}} collects monitoring data for your installation using Filebeat and Metricbeat. This data gets sent as monitoring indices to a dedicated `logging-and-metrics` deployment that is created whenever you install {{ece}} on your first host. Data is collected on every host that is part of your {{ece}} installation and includes:
1212

13-
* Logs for all core services that are a part of Elastic Cloud Enterprise and monitoring metrics for core Elastic Cloud Enterprise services, system processes on the host, and any third-party software
14-
* Logs and monitoring metrics for Elasticsearch clusters and for Kibana instances
13+
* Logs for all core services that are a part of {{ece}}
14+
* Monitoring metrics for core {{ece}} services, system processes on the host, and any third-party software
15+
* Logs and monitoring metrics for {{es}} clusters and for {{kib}} instances
1516

16-
These monitoring indices are collected in addition to the [monitoring you might have enabled for specific clusters](/deploy-manage/monitor/stack-monitoring/ece-ech-stack-monitoring.md), which also provides monitoring metrics that you can access in Kibana (note that the `logging-and-metrics` deployment is used for monitoring data from system deployments only; for non-system deployments, monitoring data must be sent to a deployment other than `logging-and-metrics`).
17+
These monitoring indices are collected in addition to the [monitoring you might have enabled for specific clusters](/deploy-manage/monitor/stack-monitoring/ece-ech-stack-monitoring.md), which also provides monitoring metrics that you can access in Kibana.
1718

18-
In this section:
19+
The `logging-and-metrics` deployment is used for monitoring data from system deployments only. For non-system deployments, monitoring data must be sent to a deployment other than `logging-and-metrics`.
1920

20-
* [Access logs and metrics](ece-monitoring-ece-access.md) - Where to find the logs and metrics that are collected.
21-
* [Capture heap dumps](../../../troubleshoot/deployments/cloud-enterprise/heap-dumps.md) - Troubleshoot instances that run out of memory.
22-
* [Capture thread dumps](../../../troubleshoot/deployments/cloud-enterprise/thread-dumps.md) - Troubleshoot instances that are having thread or CPU issues.
23-
* [Set the Retention Period for Logging and Metrics Indices](ece-monitoring-ece-set-retention.md) - Set the retention period for the indices that Elastic Cloud Enterprise collects.
21+
In this section, you'll learn the following about using ECE platform monitoring:
2422

25-
::::{important}
26-
The `logging-and-metrics` deployment is for use by your ECE installation only. You must not use this deployment to index monitoring data from your own Elasticsearch clusters or use it to index data from Beats and Logstash. Always create a separate, dedicated monitoring deployment for your own use.
27-
::::
23+
* [](ece-monitoring-ece-access.md): The types of logs and metrics that are collected for deployments, and where to find them.
24+
% where do we find logs and metrics for the installation itself?
25+
* [](/deploy-manage/monitor/orchestrators/ece-proxy-log-fields.md): The fields that are included in proxy logs. Proxy logs capture the search and indexing requests that proxies have sent to the {{es}} cluster, and requests that proxies have sent to the {{kib}} instance.
26+
* [](ece-monitoring-ece-set-retention.md): How to set the retention period for the indices that {{ece}} collects.
2827

28+
For information about troubleshooting {{ECE}} using these metrics, and guidance on capturing other diagnostic information like heap dumps and thread dumps, refer to [](/troubleshoot/deployments/cloud-enterprise/cloud-enterprise).
2929

30+
::::{important}
31+
The `logging-and-metrics` deployment is for use by your ECE installation only. You must not use this deployment to index monitoring data from your own Elasticsearch clusters or use it to index data from Beats and Logstash. Always create a separate, dedicated monitoring deployment for your own use.
32+
::::

0 commit comments

Comments
 (0)