You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-key-scenarios-to-monitor.md
+15-15Lines changed: 15 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,20 +8,20 @@ ms.reviewer: jasonh
8
8
ms.service: hdinsight
9
9
ms.custom: hdinsightactive
10
10
ms.topic: conceptual
11
-
ms.date: 09/27/2017
12
-
ms.author: maxluk
11
+
ms.date: 11/06/2018
12
+
ms.author: arindamc
13
13
---
14
14
# Monitor cluster performance
15
15
16
-
Monitoring the health and performance of an HDInsight cluster is essential for maintaining maximum performance and resource utilization. Monitoring can also help you address possible coding or cluster configuration errors.
16
+
Monitoring the health and performance of an HDInsight cluster is essential for maintaining optimal performance and resource utilization. Monitoring can also help you detect and address cluster configuration errors and user code issues.
17
17
18
-
The following sections describe how to optimize cluster loading, YARN queue efficiency, and storage accessibility.
18
+
The following sections describe how to monitor and optimize the load on your clusters, YARN queues and detect storage throttling issues.
19
19
20
-
## Cluster loading
20
+
## Monitor cluster load
21
21
22
-
Hadoop clusters should balance loading across the nodes of the cluster. This balancing prevents processing tasks from being constrained by RAM, CPU, or disk resources.
22
+
Hadoop clusters can deliver the most optimal performance when the load on cluster is evenly distributed across all the nodes. This enables the processing tasks to run without being constrained by RAM, CPU, or disk resources on individual nodes.
23
23
24
-
To get a high-level look at the nodes of your cluster and their loading, log in to the [Ambari Web UI](hdinsight-hadoop-manage-ambari.md), then select the **Hosts** tab. Your hosts are listed by their fully qualified domain names. Each host's operating status is shown by a colored health indicator:
24
+
To get a high-level look at the nodes of your cluster and their loading, sign in to the [Ambari Web UI](hdinsight-hadoop-manage-ambari.md), then select the **Hosts** tab. Your hosts are listed by their fully qualified domain names. Each host's operating status is shown by a colored health indicator:
25
25
26
26
| Color | Description |
27
27
| --- | --- |
@@ -42,29 +42,29 @@ See [Manage HDInsight clusters by using the Ambari Web UI](hdinsight-hadoop-mana
42
42
43
43
## YARN queue configuration
44
44
45
-
Hadoop has various services running across its distributed platform. YARN (Yet Another Resource Negotiator) coordinates these services, allocates cluster resources, and manages access to a common data set.
45
+
Hadoop has various services running across its distributed platform. YARN (Yet Another Resource Negotiator) coordinates these services and allocates cluster resources to ensure that any load is evenly distributed across the cluster.
46
46
47
-
YARN divides the two responsibilities of the JobTracker, resource management and job scheduling/monitoring, into two daemons: a global ResourceManager, and a per-application ApplicationMaster (AM).
47
+
YARN divides the two responsibilities of the JobTracker, resource management and job scheduling/monitoring, into two daemons: a global Resource Manager, and a per-application ApplicationMaster (AM).
48
48
49
-
The ResourceManager is a *pure scheduler*, and solely arbitrates available resources between all competing applications. The ResourceManager ensures that all resources are always in use, optimizing for various constants such as SLAs, capacity guarantees, and so forth. The ApplicationMaster negotiates resources from the ResourceManager, and works with the NodeManager(s) to execute and monitor the containers and their resource consumption.
49
+
The Resource Manager is a *pure scheduler*, and solely arbitrates available resources between all competing applications. The Resource Manager ensures that all resources are always in use, optimizing for various constants such as SLAs, capacity guarantees, and so forth. The ApplicationMaster negotiates resources from the Resource Manager, and works with the NodeManager(s) to execute and monitor the containers and their resource consumption.
50
50
51
51
When multiple tenants share a large cluster, there is competition for the cluster's resources. The CapacityScheduler is a pluggable scheduler that assists in resource sharing by queueing up requests. The CapacityScheduler also supports *hierarchical queues* to ensure that resources are shared between the sub-queues of an organization, before other applications' queues are allowed to use free resources.
52
52
53
-
YARN allows us to allocate resources to these queues, and shows you whether all of your available resources are assigned. To view information about your queues, log in to the Ambari Web UI, then select **YARN Queue Manager** from the top menu.
53
+
YARN allows us to allocate resources to these queues, and shows you whether all of your available resources are assigned. To view information about your queues, sign in to the Ambari Web UI, then select **YARN Queue Manager** from the top menu.
For a more detailed look at your queues, from the Ambari dashboard, select the **YARN** service from the list on the left. Then under the **Quick Links** dropdown menu, select **ResourceManager UI** underneath your active node.
61
+
For a more detailed look at your queues, from the Ambari dashboard, select the **YARN** service from the list on the left. Then under the **Quick Links** dropdown menu, select **Resource Manager UI** underneath your active node.
62
62
63
-

63
+

64
64
65
-
In the ResourceManager UI, select **Scheduler** from the left-hand menu. You see a list of your queues underneath *Application Queues*. Here you can see the capacity used for each of your queues, how well the jobs are distributed between them, and whether any jobs are resource-constrained.
65
+
In the Resource Manager UI, select **Scheduler** from the left-hand menu. You see a list of your queues underneath *Application Queues*. Here you can see the capacity used for each of your queues, how well the jobs are distributed between them, and whether any jobs are resource-constrained.
66
66
67
-

67
+

0 commit comments