You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: deploy-manage/monitor.md
+8-4Lines changed: 8 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -74,6 +74,13 @@ Out of the box logs and metrics tools, including ECH preconfigured logs and metr
74
74
75
75
To learn more about the health and performance tools in {{ecloud}}, refer to [](/deploy-manage/monitor/cloud-health-perf.md).
76
76
77
+
## {{kib}} task manager monitoring
78
+
79
+
```{applies_to}
80
+
stack: preview
81
+
```
82
+
The {{kib}} [task manager](/deploy-manage/distributed-architecture/kibana-tasks-management.md) has an internal monitoring mechanism to keep track of a variety of metrics, which can be consumed with either the health monitoring API or the {{kib}} server log. [Learn how to configure thresholds and consume related to {{kib}} task manager](/deploy-manage/monitor/kibana-task-manager-health-monitoring.md).
83
+
77
84
## Monitoring your orchestrator
78
85
```{applies_to}
79
86
deployment:
@@ -94,7 +101,4 @@ Elastic monitors [{{ecloud}}](/deploy-manage/deploy/elastic-cloud.md) service me
You can configure several types of logs in {{stack}} that can help you to gain insight into {{stack}} operations, diagnose issues, and track certain types of events. [Learn about the types of logs available, where to find them, and how to configure them](/deploy-manage/monitor/logging-configuration.md).
# Kibana task manager health monitoring [task-manager-health-monitoring]
14
10
15
11
@@ -18,7 +14,7 @@ This functionality is in technical preview and may be changed or removed in a fu
18
14
::::
19
15
20
16
21
-
The Task Manager has an internal monitoring mechanism to keep track of a variety of metrics, which can be consumed with either the health monitoring API or the {{kib}} server log.
17
+
The {{kib}} [Task Manager](/deploy-manage/distributed-architecture/kibana-tasks-management.md) has an internal monitoring mechanism to keep track of a variety of metrics, which can be consumed with either the health monitoring API or the {{kib}} server log.
22
18
23
19
The health monitoring API provides a reliable endpoint that can be monitored. Consuming this endpoint doesn’t cause additional load, but rather returns the latest health checks made by the system. This design enables consumption by external monitoring services at a regular cadence without additional load to the system.
## Consuming health stats [task-manager-consuming-health-stats]
61
57
62
-
The health API is best consumed by via the `/api/task_manager/_health` endpoint.
58
+
The health API is best consumed using the `/api/task_manager/_health` endpoint.
63
59
64
60
Additionally, there are two ways to consume these metrics:
65
61
66
-
**Debug logging**
62
+
### Debug logging
63
+
```{applies_to}
64
+
deployment:
65
+
self:
66
+
```
67
67
68
-
The metrics are logged in the {{kib}} `DEBUG` logger at a regular cadence. To enable Task Manager debug logging in your {{kib}} instance, add the following to your `kibana.yml`:
68
+
In self-managed deployments, you can configure health stats to be logged in the {{kib}} `DEBUG` logger at a regular cadence. To enable Task Manager debug logging in your {{kib}} instance, add the following to your `kibana.yml`:
69
69
70
70
```yaml
71
71
logging:
@@ -77,7 +77,7 @@ logging:
77
77
78
78
These stats are logged based on the number of milliseconds set in your [`xpack.task_manager.poll_interval`](kibana://reference/configuration-reference/task-manager-settings.md#task-manager-settings) setting, which could add substantial noise to your logs. Only enable this level of logging temporarily.
79
79
80
-
**Automatic logging**
80
+
### Automatic logging
81
81
82
82
By default, the health API runs at a regular cadence, and each time it runs, it attempts to self evaluate its performance. If this self evaluation yields a potential problem, a message will log to the {{kib}} server log. In addition, the health API will look at how long tasks have waited to start (from when they were scheduled to start). If this number exceeds a configurable threshold ([`xpack.task_manager.monitored_stats_health_verbose_log.warn_delayed_task_start_in_seconds`](kibana://reference/configuration-reference/task-manager-settings.md#task-manager-settings)), the same message as above will log to the {{kib}} server log.
83
83
@@ -92,9 +92,9 @@ If this message appears, set [`xpack.task_manager.monitored_stats_health_verbose
92
92
93
93
## Making sense of Task Manager health stats [making-sense-of-task-manager-health-stats]
94
94
95
-
The health monitoring API exposes three sections: `configuration`, `workload` and `runtime`:
95
+
The health monitoring API exposes the following sections:
96
96
97
-
|||
97
+
|Section | Description|
98
98
| --- | --- |
99
99
| Configuration | This section summarizes the current configuration of Task Manager. This includes dynamic configurations that change over time, such as `poll_interval` and `max_workers`, which can adjust in reaction to changing load on the system. |
100
100
| Workload | This section summarizes the work load across the cluster, including the tasks in the system, their types, and current status. |
Copy file name to clipboardExpand all lines: deploy-manage/monitor/logging-configuration.md
+63-26Lines changed: 63 additions & 26 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,57 +6,74 @@ applies_to:
6
6
eck: all
7
7
self: all
8
8
---
9
-
# Logging configuration
9
+
# Logging
10
10
11
-
% What needs to be done: Write from scratch
11
+
You can configure several types of logs in {{stack}} that can help you to gain insight into {{stack}} operations, diagnose issues, and track certain types of events.
## For {{es}} [extra-logging-features-elasticsearch]
16
16
17
+
* Application and component logging: Logs messages related to running {{es}}.
18
+
19
+
You can [configure the log level for {{es}}](/deploy-manage/monitor/logging-configuration/update-elasticsearch-logging-levels.md), and, in self-managed clusters, [configure underlying Log4j settings](/deploy-manage/monitor/logging-configuration/elasticsearch-log4j-configuration-self-managed.md) to customize logging behavior.
20
+
*[Deprecation logging](/deploy-manage/monitor/logging-configuration/elasticsearch-deprecation-logs.md): Deprecation logs record a message to the {{es}} log directory when you use deprecated {{es}} functionality. You can use the deprecation logs to update your application before upgrading {{es}} to a new major version.
21
+
*[Audit logging](/deploy-manage/security/logging-configuration/enabling-audit-logs.md): Logs security-related events on your deployment.
22
+
*[Slow query and index logging](elasticsearch://reference/elasticsearch/index-settings/slow-log.md): Helps find and debug slow queries and indexing.
17
23
18
-
## Logging features [ECE/ECH][extra-logging-features]
24
+
## For {{kib}} [extra-logging-features-kibana]
19
25
20
-
When shipping logs to a monitoring deployment there are more logging features available to you. These features include:
26
+
*[Audit logging](/deploy-manage/security/logging-configuration/enabling-audit-logs.md): Logs security-related events on your deployment.
21
27
28
+
## Access {{kib}} and {{es}} logs
22
29
23
-
### For {{es}} [extra-logging-features-elasticsearch]
30
+
The way that you access your logs differs depending on your deployment method.
24
31
25
-
*[Audit logging](/deploy-manage/security/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
26
-
*[Slow query and index logging](elasticsearch://reference/elasticsearch/index-settings/slow-log.md) - helps find and debug slow queries and indexing
After you’ve enabled log delivery on your deployment, you can [add the Elasticsearch user settings](/deploy-manage/deploy/cloud-enterprise/edit-stack-settings.md) to enable these features.
34
+
Access your logs using one of the following options:
30
35
36
+
* All orchestrated deployments: [](/deploy-manage/monitor/stack-monitoring.md)
37
+
* {{ech}}: [Preconfigured logs and metrics](/deploy-manage/monitor/cloud-health-perf.md#ec-es-health-preconfigured)
*[Audit logging](/deploy-manage/security/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
42
+
#### {{kib}}
35
43
36
-
After you’ve enabled log delivery on your deployment, you can [add the {{kib}} user settings](/deploy-manage/deploy/cloud-enterprise/edit-stack-settings.md) to enable this feature.
44
+
If you run {{kib}} as a service, the default location of the logs varies based on your platform and installation method:
37
45
46
+
:::::::{tab-set}
38
47
39
-
### Other components [extra-logging-features-enterprise-search]
48
+
::::::{tab-item} Docker
49
+
On [Docker](../../deploy/self-managed/install-elasticsearch-with-docker.md), log messages go to the console and are handled by the configured Docker logging driver. To access logs, run `docker logs`.
50
+
::::::
40
51
41
-
Enabling log collection also supports collecting and indexing the following types of logs from other components in your deployments:
52
+
::::::{tab-item} Debian (APT) and RPM
53
+
For [Debian](../../deploy/self-managed/install-elasticsearch-with-debian-package.md) and [RPM](../../deploy/self-managed/install-elasticsearch-with-rpm.md) installations, {{es}} writes logs to `/var/log/kibana`.
54
+
::::::
42
55
43
-
**APM**
56
+
::::::{tab-item} macOS and Linux
57
+
For [macOS and Linux `.tar.gz`](../../deploy/self-managed/install-elasticsearch-from-archive-on-linux-macos.md) installations, {{es}} writes logs to `$KIBANA_HOME/logs`.
44
58
45
-
*`apm*.log*`
59
+
Files in `$KIBANA_HOME` risk deletion during an upgrade. In production, you should configure a [different location for your logs](/deploy-manage/monitor/logging-configuration/kib-self-managed-logging.md).
60
+
::::::
46
61
47
-
**Fleet and Elastic Agent**
62
+
::::::{tab-item} Windows .zip
63
+
For [Windows `.zip`](../../deploy/self-managed/install-elasticsearch-with-zip-on-windows.md) installations, {{es}} writes logs to `%KIBANA_HOME%\logs`.
48
64
49
-
*`fleet-server-json.log-*`
50
-
*`elastic-agent-json.log-*`
65
+
Files in `%KIBANA_HOME%` risk deletion during an upgrade. In production, you should configure a [different location for your logs](/deploy-manage/monitor/logging-configuration/kib-self-managed-logging.md).
66
+
::::::
51
67
52
-
The `*` indicates that we also index the archived files of each type of log.
68
+
:::::::
53
69
54
-
Check the respective product documentation for more information about the logging capabilities of each product.
70
+
If you run {{kib}} from the command line, {{es}} prints logs to the standard output (`stdout`).
55
71
56
-
=====
72
+
You can also consume logs using [stack monitoring](/deploy-manage/monitor/stack-monitoring/kibana-monitoring-self-managed.md).
57
73
74
+
#### {{es}}
58
75
59
-
You can use {{es}}'s application logs to monitor your cluster and diagnose issues. If you run {{es}} as a service, the default location of the logs varies based on your platform and installation method:
76
+
If you run {{es}} as a service, the default location of the logs varies based on your platform and installation method:
60
77
61
78
:::::::{tab-set}
62
79
@@ -81,4 +98,24 @@ Files in `%ES_HOME%` risk deletion during an upgrade. In production, we strongly
81
98
::::::
82
99
83
100
:::::::
84
-
If you run {{es}} from the command line, {{es}} prints logs to the standard output (`stdout`).
101
+
102
+
If you run {{es}} from the command line, {{es}} prints logs to the standard output (`stdout`).
103
+
104
+
You can also consume logs using [stack monitoring](/deploy-manage/monitor/stack-monitoring/elasticsearch-monitoring-self-managed.md).
105
+
106
+
## Other components [extra-logging-features-enterprise-search]
107
+
108
+
You can also collect and index the following types of logs from other components in your deployments:
Copy file name to clipboardExpand all lines: deploy-manage/monitor/logging-configuration/elasticsearch-deprecation-logs.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ applies_to:
11
11
12
12
# Elasticsearch deprecation logs [logging]
13
13
14
-
{{es}} also writes deprecation logs to the log directory. These logs record a message when you use deprecated {{es}} functionality. You can use the deprecation logs to update your application before upgrading {{es}} to a new major version.
14
+
{{es}} writes deprecation logs to the [log directory](/deploy-manage/monitor/logging-configuration#access-kib-and-es-logs). These logs record a message when you use deprecated {{es}} functionality. You can use the deprecation logs to update your application before upgrading {{es}} to a new major version.
15
15
16
16
:::{tip}
17
17
You can also access deprecation warnings in the [upgrade assistant](/deploy-manage/upgrade/prepare-to-upgrade/upgrade-assistant.md).
0 commit comments