You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: deploy-manage/production-guidance/kibana-traffic-scaling-considerations.md
+19-16Lines changed: 19 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,17 +22,18 @@ On [serverless](../deploy/elastic-cloud/serverless.md) scaling {{kib}} is fully
22
22
23
23
{{kib}}'s HTTP traffic is diverse and can be unpredictable. Traffic includes serving static assets like files, processing large search responses from {{es}} and managing CRUD operations against complex domain objects like SLOs. The scale of the load created by each of these kinds of traffic will vary depending on your usage patterns. While difficult to predict, there are 2 important aspects to consider when provisioning CPU and memory resources for your {{kib}} instances:
24
24
25
-
1. Concurrency: how many users do you expect to be interacting with {{kib}} simultaneously which is largely **CPUbound**
26
-
2. Request and response size: how large (usually measured in bytes) are the requests and responses you expect to {{kib}} to service which largely **memorybound**
25
+
1. Concurrency: how many users do you expect to be interacting with {{kib}} simultaneously, which is largely **CPU-bound**
26
+
2. Request and response size: how large (usually measured in bytes) are the requests and responses you expect {{kib}} to service, which is largely **memory-bound**
27
27
28
28
::::{important}
29
-
The nature of traffic is not only diverse, but also unpredictable. Traffic to {{kib}} often comes in short bursts or spikes that can overwhelm an underprovisioned {{kib}}. In production environments an overwhelmed {{kib}} will typically manifest as 502 or 503 error responses from {{kib}}.
29
+
30
+
The nature of traffic is not only diverse, but also unpredictable. Traffic to {{kib}} often comes in short bursts or spikes that can overwhelm an underprovisioned {{kib}}. In production environments, an overwhelmed {{kib}} will typically manifest as 502 or 503 error responses from {{kib}}.
30
31
31
32
A valuable strategy known as load balancing helps to mitigate this bursty nature of traffic by horizontally scaling your {{kib}} deployments and improving {{kib}}'s availability. See the guide on [load balancing traffic](./kibana-load-balance-traffic.md). The rest of this guide will focus on provisioning CPU and memory (also known as vertically scaling) a single Kibana for handling your traffic load, but is not a replacement for load balancing traffic.
32
33
::::
33
34
34
35
::::{important}
35
-
CPU and memory boundedness often interact in important ways. If CPU-bound activity is reaching it's limit memory pressure will likely increase as {{kib}} has less time for activities like garbage collection. If memory-bound activity is reaching it's limit there may be more CPU work to free claimed memory, increasing CPU pressure.
36
+
CPU and memory boundedness often interact in important ways. If CPU-bound activity is reaching its limit, memory pressure will likely increase as {{kib}} has less time for activities like garbage collection. If memory-bound activity is reaching its limit, there may be more CPU work to free claimed memory, increasing CPU pressure.
36
37
::::
37
38
38
39
### Answer the following questions before scaling Kibana up or down [_before_sizing_kibana]
@@ -43,19 +44,21 @@ Follow [the production guidance for {{es}} first](./elasticsearch-in-production-
43
44
44
45
#### What requests is {{kib}} sending to {{es}}?
45
46
46
-
In user interfaces like Dashboards or Discover one can see the full query that {{kib}} is sending to {{es}}. This is a good way to get an idea of the volume of data and work a {{kib}} visualization or dashboard is creating for {{es}}. Dashboards with many visualizations will generate higher load for {{es}} and {{kib}}.
47
+
In user interfaces like Dashboards or Discover, one can see the full query that {{kib}} is sending to {{es}}. This is a good way to get an idea of the volume of data and work a {{kib}} visualization or dashboard is creating for {{es}}. Dashboards with many visualizations will generate higher load for {{es}} and {{kib}}.
47
48
48
49
### A simple sizing strategy
49
50
50
-
If you know max number of expected concurrent users start {{kib}} on **1 CPU** and **1.5GB** of memory. This should comfortably serve a set of 10 concurrent users performing analytics activities like browsing dashboards. If you are experiencing performance issues, adding an additional **0.5 CPUs** and **1.5GB** per 10 concurrent should ensure {{kib}} is not resource starved for common analytics usage.
51
+
Follow this strategy if you know the max number of expected concurrent users.
52
+
53
+
Start {{kib}} on **1 CPU** and **1.5GB** of memory. This should comfortably serve a set of 10 concurrent users performing analytics activities like browsing dashboards. If you are experiencing performance issues, adding an additional **0.5 CPUs** and **1.5GB** per 10 concurrent users should ensure {{kib}} is not resource-starved for common analytics usage.
51
54
52
55
::::{important}
53
-
This advice does not apply to scaling {{kib}} for task manager. If you intend to use {{kib}} alerting capabilites see [task manager scaling guidance](./kibana-task-manager-scaling-considerations.md).
56
+
This advice does not apply to scaling {{kib}} for task manager. If you intend to use {{kib}} alerting capabilities see [task manager scaling guidance](./kibana-task-manager-scaling-considerations.md).
54
57
::::
55
58
56
-
**{{ece}}, {{ech}} and {{eck}}** users can adjust {{kib}}'s memory by viewing their deployment and editing the {{kib}} instance's resource configuration. Note: size increments are predetermined and may not fit this simple sizing strategy exactly.
59
+
**{{ece}}, {{ech}}, and {{eck}}** users can adjust {{kib}}'s memory by viewing their deployment and editing the {{kib}} instance's resource configuration. Note: size increments are predetermined and may not fit this simple sizing strategy exactly.
57
60
58
-
**Self-managed** users must provision memory to the host that {{kib}} is running on as well as configure allocated heap, see[the guidance on configuring {{kib}} memory](./kibana-configure-memory.md). **Note:** Node.js suggests allocating 80% of available host memory to heap, assuming that Kibana is the only server process running on the (virtual) host. This allows for memory resources to be used for other activities, for example: allowing for HTTP sockets to allocated.
61
+
**Self-managed** users must provision memory to the host that {{kib}} is running on as well as configure allocated heap. See[the guidance on configuring {{kib}} memory](./kibana-configure-memory.md). **Note:** Node.js suggests allocating 80% of available host memory to heap, assuming that Kibana is the only server process running on the (virtual) host. This allows for memory resources to be used for other activities, for example: allowing for HTTP sockets to be allocated.
@@ -71,27 +74,27 @@ In order to understand the impact of your usage patterns on **a {{kib}} instance
71
74
72
75
The rest of this guide will assume you have visibility into the following important metrics for a {{kib}} instance:
73
76
74
-
1. Event loop delay (ELD) in milliseconds - this is a Node.js concept that roughly translates to: the number of milliseconds by which processing of events is delayed due to CPUintensive activities
77
+
1. Event loop delay (ELD) in milliseconds - this is a Node.js concept that roughly translates to the number of milliseconds by which processing of events is delayed due to CPU-intensive activities
75
78
2. Heap size in bytes - the amount of bytes currently held in memory dedicated to {{kib}}'s heap space
76
79
3. HTTP connections - the number of sockets that the Kibana server has open
77
80
78
81
##### CPU [kibana-traffic-load-cpu-sizing]
79
82
80
83
Event loop delay (ELD) is an important metric for understanding whether Kibana is engaged in CPU-bound activity.
81
84
82
-
**As a general target ELD should be below 200ms 95% of the time**. Higher delays may mean {{kib}} is CPUstarved. Sporadic increases above 200ms may mean that Kibana is periodically processing CPUintensive activities like large responses from Elasticsearch whereas consistently high ELD may mean Kibana is struggling to service tasks and requests.
85
+
**As a general target, ELD should be below 200ms 95% of the time**. Higher delays may mean {{kib}} is CPU-starved. Sporadic increases above 200ms may mean that Kibana is periodically processing CPU-intensive activities like large responses from Elasticsearch, whereas consistently high ELD may mean Kibana is struggling to service tasks and requests.
83
86
84
-
Consider the impact of ELD on user experience. If users are able to use {{kib}} without the frustration that comes from a blocked CPU provisioning additional CPU resources will not be impactful. Monitoring ELD over time is a solid strategy for knowing when additional CPU resource is needed.
87
+
Consider the impact of ELD on user experience. If users are able to use {{kib}} without the frustration that comes from a blocked CPU, provisioning additional CPU resources will not be impactful. Monitoring ELD over time is a solid strategy for knowing when additional CPU resource is needed.
85
88
86
-
**{{ece}}, {{ech}} and {{eck}}** users can adjust {{kib}}'s CPU and memory by viewing their deployment and editing the {{kib}} instance's resource configuration.
89
+
**{{ece}}, {{ech}}, and {{eck}}** users can adjust {{kib}}'s CPU and memory by viewing their deployment and editing the {{kib}} instance's resource configuration.
87
90
88
91
**Self-managed** users are responsible for managing CPU.
89
92
90
93
##### Memory [kibana-traffic-load-memory-sizing]
91
94
92
-
Heap size relative is an important metric to track. If {{kib}}'s heap size grows beyond the heap limit {{kib}} will crash. By monitoring heap size you can help ensure that {{kib}} has enough memory available.
95
+
Heap size is an important metric to track. If {{kib}}'s heap size grows beyond the heap limit, {{kib}} will crash. By monitoring heap size, you can help ensure that {{kib}} has enough memory available.
93
96
94
-
**{{ece}}, {{ech}} and {{eck}}** users can adjust {{kib}}'s CPU and memory by viewing their deployment and editing the {{kib}} instance's resource configuration.
97
+
**{{ece}}, {{ech}}, and {{eck}}** users can adjust {{kib}}'s CPU and memory by viewing their deployment and editing the {{kib}} instance's resource configuration.
95
98
96
-
**Self-managed** users must provision memory to the host that {{kib}} is running on as well as configure allocated heap, see[the guidance on configuring {{kib}} memory](./kibana-configure-memory.md). **Note:** Node.js suggests allocating 80% of available memory to heap. This allows for memory resources to be used for other activities, for example: allowing for HTTP sockets to allocated.
99
+
**Self-managed** users must provision memory to the host that {{kib}} is running on as well as configure allocated heap. See[the guidance on configuring {{kib}} memory](./kibana-configure-memory.md). **Note:** Node.js suggests allocating 80% of available memory to heap. This allows for memory resources to be used for other activities, for example: allowing for HTTP sockets to be allocated.
0 commit comments