You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
1. Connect to the HDInsight cluster by using a Secure Shell (SSH) client. For more information, see [Additional reading](#additional-reading-2).
61
+
1. Connect to the HDInsight cluster by using a Secure Shell (SSH) client. For more information, see [Further reading](#additional-reading-2).
62
62
63
63
1. To list all the application IDs of the YARN applications that are currently running, run the following command:
64
64
@@ -131,22 +131,22 @@ These changes are visible immediately on the YARN Scheduler UI.
131
131
132
132
### Yarn UI isn't loading
133
133
134
-
If your YARN UI isn't loading or is unreachable, and it returns "HTTP Error 502.3 - Bad Gateway," it highly indicates your ResourceManager service is unhealthy. To mitigate the issue, follow these steps:
134
+
If your YARN UI isn't loading or is unreachable, and it returns "HTTP Error 502.3 - Bad Gateway," it highly indicates your Resource Manager service is unhealthy. To mitigate the issue, follow these steps:
135
135
136
-
1. Go to **Ambari UI** > **YARN** > **SUMMARY** and check to see if only the active ResourceManager is in the **Started** state. If not, try to mitigate by restarting the unhealthy or stopped ResourceManager.
137
-
2. If step 1 doesn't resolve the issue, SSH the active ResourceManager head node and check the garbage collection status using `jstat -gcutil <ResourceManager pid> 1000 100`. If you see the **FGCT** increase significantly in just a few seconds, it indicates ResourceManager is busy in *Full GC*, and is unable to process the other requests.
138
-
3. Go to **Ambari UI** > **YARN** > **CONFIGS** > **Advanced** and increase `ResourceManager java heap size`.
136
+
1. Go to **Ambari UI** > **YARN** > **SUMMARY** and check to see if only the active Resource Manager is in the **Started** state. If not, try to mitigate by restarting the unhealthy or stopped Resource Manager.
137
+
2. If step 1 doesn't resolve the issue, SSH the active Resource Manager head node and check the garbage collection status using `jstat -gcutil <Resource Manager pid> 1000 100`. If you see the **FGCT** increase significantly in just a few seconds, it indicates Resource Manager is busy in *Full GC*, and is unable to process the other requests.
138
+
3. Go to **Ambari UI** > **YARN** > **CONFIGS** > **Advanced** and increase `Resource Manager java heap size`.
139
139
4. Restart required services in Ambari UI.
140
140
141
141
### Both resource managers are in standby
142
142
143
-
1. Check ResourceManager log to see if below similar error exists.
143
+
1. Check Resource Manager log to see if similar error exists.
144
144
```
145
145
Service RMActiveServices failed in state STARTED; cause: org.apache.hadoop.service.ServiceStateException: com.google.protobuf.InvalidProtocolBufferException: Could not obtain block: BP-452067264-10.0.0.16-1608006815288:blk_1074235266_494491 file=/yarn/node-labels/nodelabel.mirror
146
146
```
147
147
2. If the error exists, check to see if some files are under replication or if there are missing blocks in the HDFS. You can run `hdfs fsck hdfs://mycluster/`
148
148
149
-
3. Run `hdfs fsck hdfs://mycluster/ -delete` to forcefully clean up the HDFS and to get rid of the standby RM issue. Alternatively, run [PatchYarnNodeLabel](https://hdiconfigactions.blob.core.windows.net/hadoopcorepatchingscripts/PatchYarnNodeLabel.sh) on one of headnodes to patch the cluster.
149
+
3. Run `hdfs fsck hdfs://mycluster/ -delete` too forcefully cleanup the HDFS and to get rid of the standby RM issue. Alternatively, run [PatchYarnNodeLabel](https://hdiconfigactions.blob.core.windows.net/hadoopcorepatchingscripts/PatchYarnNodeLabel.sh) on one of headnodes to patch the cluster.
0 commit comments