Skip to content

Commit 1f5bd28

Browse files
committed
ts_hbase7
1 parent 7ce84c4 commit 1f5bd28

File tree

2 files changed

+95
-9
lines changed

2 files changed

+95
-9
lines changed

articles/hdinsight/hbase/TOC.yml

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -25,14 +25,6 @@
2525
href: ../hdinsight-component-versioning.md?toc=/azure/hdinsight/hbase/toc.json&bc=/azure/hdinsight/hbase/breadcrumb/toc.json
2626
- name: HDInsight 4.0
2727
href: ../hdinsight-version-release.md?toc=/azure/hdinsight/hbase/toc.json&bc=/azure/hdinsight/hbase/breadcrumb/toc.json
28-
- name: Troubleshoot
29-
items:
30-
- name: Apache HBase REST not responding
31-
href: ./hbase-troubleshoot-rest-not-spending.md
32-
- name: Pegged CPU on region server
33-
href: ./hbase-troubleshoot-pegged-cpu-region-server.md
34-
- name: Timeouts with 'hbase hbck' command
35-
href: ./hbase-troubleshoot-timeouts-hbase-hbck.md
3628
- name: How to
3729
items:
3830
- name: Use cluster storage
@@ -167,8 +159,17 @@
167159
- name: OS patching for HDInsight cluster
168160
href: ../hdinsight-os-patching.md?toc=/azure/hdinsight/hbase/toc.json&bc=/azure/hdinsight/hbase/breadcrumb/toc.json
169161
- name: Troubleshoot
170-
href: ../hdinsight-troubleshoot-guide.md?toc=/azure/hdinsight/hbase/toc.json&bc=/azure/hdinsight/hbase/breadcrumb/toc.json
171162
items:
163+
- name: Troubleshoot resources
164+
href: ../hdinsight-troubleshoot-guide.md?toc=/azure/hdinsight/hbase/toc.json&bc=/azure/hdinsight/hbase/breadcrumb/toc.json
165+
- name: Issues with region servers
166+
href: ./hbase-troubleshoot-unassigned-regions.md
167+
- name: Apache HBase REST not responding
168+
href: ./hbase-troubleshoot-rest-not-spending.md
169+
- name: Pegged CPU on region server
170+
href: ./hbase-troubleshoot-pegged-cpu-region-server.md
171+
- name: Timeouts with 'hbase hbck' command
172+
href: ./hbase-troubleshoot-timeouts-hbase-hbck.md
172173
- name: Troubleshoot a slow or failing HDInsight cluster
173174
href: ../hdinsight-troubleshoot-failed-cluster.md?toc=/azure/hdinsight/hbase/toc.json&bc=/azure/hdinsight/hbase/breadcrumb/toc.json
174175
- name: Apache Hadoop HDFS troubleshooting
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
---
2+
title: Issues with region servers in Azure HDInsight
3+
description: Issues with region servers in Azure HDInsight
4+
ms.service: hdinsight
5+
ms.topic: troubleshooting
6+
author: hrasheed-msft
7+
ms.author: hrasheed
8+
ms.date: 08/07/2019
9+
---
10+
11+
# Issues with region servers in Azure HDInsight
12+
13+
This article describes troubleshooting steps and possible resolutions for issues when interacting with Azure HDInsight clusters.
14+
15+
## Scenario: Unassigned regions
16+
17+
### Issue
18+
19+
When running `hbase hbck` command, you see an error message similar to:
20+
21+
```
22+
multiple regions being unassigned or holes in the chain of regions
23+
```
24+
25+
From the Apache HBase Master UI, it can be seen that the count of regions being unbalanced across all the region servers.
26+
27+
### Cause
28+
29+
Holes may be the result of offline regions.
30+
31+
### Resolution
32+
33+
Fix the assignments. Follow the steps below to bring the unassigned regions back to normal state:
34+
35+
1. Sign in to HDInsight HBase cluster using SSH.
36+
37+
1. Run `hbase zkcli` command to connect with zookeeper shell.
38+
39+
1. Run `rmr /hbase/regions-in-transition` or `rmr /hbase-unsecure/regions-in-transition` command.
40+
41+
1. Exit zookeeper shell by using `exit` command.
42+
43+
1. Open Ambari UI and restart Active HBase Master service from Ambari.
44+
45+
1. Run `hbase hbck` command again (without any further options). Check the output and ensure that all regions are being assigned.
46+
47+
---
48+
49+
## Scenario: Dead region servers
50+
51+
### Issue
52+
53+
Region servers fail to start.
54+
55+
### Cause
56+
57+
Multiple splitting WAL directories.
58+
59+
1. Get list of current wals: `hadoop fs -ls -R /hbase/WALs/ > /tmp/wals.out`.
60+
61+
1. Inspect the `wals.out` file. If there are too many splitting directories (starting with *-splitting), the region server is probably failing because of these directories.
62+
63+
### Resolution
64+
65+
1. Stop HBase from Ambari portal.
66+
67+
1. Execute `hadoop fs -ls -R /hbase/WALs/ > /tmp/wals.out` to get fresh list of WALs.
68+
69+
1. Move the *-splitting directories to a temporary folder, `splitWAL`, and delete the *-splitting directories.
70+
71+
1. Execute `hbase zkcli` command to connect with zookeeper shell.
72+
73+
1. Execute `rmr /hbase-unsecure/splitWAL`.
74+
75+
1. Restart HBase service.
76+
77+
## Next steps
78+
79+
If you didn't see your problem or are unable to solve your issue, visit one of the following channels for more support:
80+
81+
* Get answers from Azure experts through [Azure Community Support](https://azure.microsoft.com/support/community/).
82+
83+
* Connect with [@AzureSupport](https://twitter.com/azuresupport) - the official Microsoft Azure account for improving customer experience. Connecting the Azure community to the right resources: answers, support, and experts.
84+
85+
* If you need more help, you can submit a support request from the [Azure portal](https://portal.azure.com/?#blade/Microsoft_Azure_Support/HelpAndSupportBlade/). Select **Support** from the menu bar or open the **Help + support** hub. For more detailed information, review [How to create an Azure support request](https://docs.microsoft.com/azure/azure-supportability/how-to-create-azure-support-request). Access to Subscription Management and billing support is included with your Microsoft Azure subscription, and Technical Support is provided through one of the [Azure Support Plans](https://azure.microsoft.com/support/plans/).

0 commit comments

Comments
 (0)