Skip to content

Commit 7aa256b

Browse files
authored
Merge pull request #84674 from dagiro/ts_hbase7
ts_hbase7
2 parents cefc13e + 3254c2c commit 7aa256b

File tree

2 files changed

+87
-0
lines changed

2 files changed

+87
-0
lines changed

articles/hdinsight/hbase/TOC.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,8 @@
162162
items:
163163
- name: Troubleshoot resources
164164
href: ../hdinsight-troubleshoot-guide.md?toc=/azure/hdinsight/hbase/toc.json&bc=/azure/hdinsight/hbase/breadcrumb/toc.json
165+
- name: Issues with region servers
166+
href: ./hbase-troubleshoot-unassigned-regions.md
165167
- name: Apache Phoenix connectivity issues
166168
href: ./hbase-troubleshoot-phoenix-connectivity.md
167169
- name: Apache HBase REST not responding
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
---
2+
title: Issues with region servers in Azure HDInsight
3+
description: Issues with region servers in Azure HDInsight
4+
ms.service: hdinsight
5+
ms.topic: troubleshooting
6+
author: hrasheed-msft
7+
ms.author: hrasheed
8+
ms.date: 08/07/2019
9+
---
10+
11+
# Issues with region servers in Azure HDInsight
12+
13+
This article describes troubleshooting steps and possible resolutions for issues when interacting with Azure HDInsight clusters.
14+
15+
## Scenario: Unassigned regions
16+
17+
### Issue
18+
19+
When running `hbase hbck` command, you see an error message similar to:
20+
21+
```
22+
multiple regions being unassigned or holes in the chain of regions
23+
```
24+
25+
From the Apache HBase Master UI, it can be seen that the count of regions being unbalanced across all the region servers.
26+
27+
### Cause
28+
29+
Holes may be the result of offline regions.
30+
31+
### Resolution
32+
33+
Fix the assignments. Follow the steps below to bring the unassigned regions back to normal state:
34+
35+
1. Sign in to HDInsight HBase cluster using SSH.
36+
37+
1. Run `hbase zkcli` command to connect with zookeeper shell.
38+
39+
1. Run `rmr /hbase/regions-in-transition` or `rmr /hbase-unsecure/regions-in-transition` command.
40+
41+
1. Exit zookeeper shell by using `exit` command.
42+
43+
1. Open Ambari UI and restart Active HBase Master service from Ambari.
44+
45+
1. Run `hbase hbck` command again (without any further options). Check the output and ensure that all regions are being assigned.
46+
47+
---
48+
49+
## Scenario: Dead region servers
50+
51+
### Issue
52+
53+
Region servers fail to start.
54+
55+
### Cause
56+
57+
Multiple splitting WAL directories.
58+
59+
1. Get list of current wals: `hadoop fs -ls -R /hbase/WALs/ > /tmp/wals.out`.
60+
61+
1. Inspect the `wals.out` file. If there are too many splitting directories (starting with *-splitting), the region server is probably failing because of these directories.
62+
63+
### Resolution
64+
65+
1. Stop HBase from Ambari portal.
66+
67+
1. Execute `hadoop fs -ls -R /hbase/WALs/ > /tmp/wals.out` to get fresh list of WALs.
68+
69+
1. Move the *-splitting directories to a temporary folder, `splitWAL`, and delete the *-splitting directories.
70+
71+
1. Execute `hbase zkcli` command to connect with zookeeper shell.
72+
73+
1. Execute `rmr /hbase-unsecure/splitWAL`.
74+
75+
1. Restart HBase service.
76+
77+
## Next steps
78+
79+
If you didn't see your problem or are unable to solve your issue, visit one of the following channels for more support:
80+
81+
* Get answers from Azure experts through [Azure Community Support](https://azure.microsoft.com/support/community/).
82+
83+
* Connect with [@AzureSupport](https://twitter.com/azuresupport) - the official Microsoft Azure account for improving customer experience. Connecting the Azure community to the right resources: answers, support, and experts.
84+
85+
* If you need more help, you can submit a support request from the [Azure portal](https://portal.azure.com/?#blade/Microsoft_Azure_Support/HelpAndSupportBlade/). Select **Support** from the menu bar or open the **Help + support** hub. For more detailed information, review [How to create an Azure support request](https://docs.microsoft.com/azure/azure-supportability/how-to-create-azure-support-request). Access to Subscription Management and billing support is included with your Microsoft Azure subscription, and Technical Support is provided through one of the [Azure Support Plans](https://azure.microsoft.com/support/plans/).

0 commit comments

Comments
 (0)