You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hbase/apache-hbase-migrate-new-version.md
+22-20Lines changed: 22 additions & 20 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,27 +2,26 @@
2
2
title: Migrate an HBase cluster to a new version - Azure HDInsight
3
3
description: How to migrate Apache HBase clusters to a newer version in Azure HDInsight.
4
4
author: ashishthaps
5
+
ms.author: ashishth
5
6
ms.reviewer: jasonh
6
7
ms.service: hdinsight
7
-
ms.custom: hdinsightactive
8
8
ms.topic: conceptual
9
-
ms.date: 12/05/2019
10
-
ms.author: ashishth
9
+
ms.custom: hdinsightactive
10
+
ms.date: 01/02/2020
11
11
---
12
12
13
13
# Migrate an Apache HBase cluster to a new version
14
14
15
15
This article discusses the steps required to update your Apache HBase cluster on Azure HDInsight to a newer version.
16
16
17
-
> [!NOTE]
18
-
> The downtime while upgrading should be minimal, on the order of minutes. This downtime is caused by the steps to flush all in-memory data, then the time to configure and restart the services on the new cluster. Your results will vary, depending on the number of nodes, amount of data, and other variables.
17
+
The downtime while upgrading should be minimal, on the order of minutes. This downtime is caused by the steps to flush all in-memory data, then the time to configure and restart the services on the new cluster. Your results will vary, depending on the number of nodes, amount of data, and other variables.
19
18
20
19
## Review Apache HBase compatibility
21
20
22
21
Before upgrading Apache HBase, ensure the HBase versions on the source and destination clusters are compatible. For more information, see [Apache Hadoop components and versions available with HDInsight](../hdinsight-component-versioning.md).
23
22
24
23
> [!NOTE]
25
-
> We highly recommend that you review the version compatibility matrix in the [HBase book](https://hbase.apache.org/book.html#upgrading).
24
+
> We highly recommend that you review the version compatibility matrix in the [HBase book](https://hbase.apache.org/book.html#upgrading). Any breaking incompatibilities should be described in the HBase version release notes.
26
25
27
26
Here is an example version compatibility matrix. Y indicates compatibility and N indicates a potential incompatibility:
28
27
@@ -40,20 +39,17 @@ Here is an example version compatibility matrix. Y indicates compatibility and N
40
39
| Dependency compatibility | N | Y | Y |
41
40
| Operational compatibility | N | N | Y |
42
41
43
-
> [!NOTE]
44
-
> Any breaking incompatibilities should be described in the HBase version release notes.
45
-
46
42
## Upgrade with same Apache HBase major version
47
43
48
44
To upgrade your Apache HBase cluster on Azure HDInsight, complete the following steps:
49
45
50
46
1. Make sure that your application is compatible with the new version, as shown in the HBase compatibility matrix and release notes. Test your application in a cluster running the target version of HDInsight and HBase.
51
47
52
-
2.[Set up a new destination HDInsight cluster](../hdinsight-hadoop-provision-linux-clusters.md) using the same storage account, but with a different container name:
48
+
1.[Set up a new destination HDInsight cluster](../hdinsight-hadoop-provision-linux-clusters.md) using the same storage account, but with a different container name:
53
49
54
50

55
51
56
-
3. Flush your source HBase cluster, which is the cluster you are upgrading. HBase writes incoming data to an in-memory store, called a _memstore_. After the memstore reaches a certain size, HBase flushes it to disk for long-term storage in the cluster's storage account. When deleting the old cluster, the memstores are recycled, potentially losing data. To manually flush the memstore for each table to disk, run the following script. The latest version of this script is on Azure's [GitHub](https://raw.githubusercontent.com/Azure/hbase-utils/master/scripts/flush_all_tables.sh).
52
+
1. Flush your source HBase cluster, which is the cluster you're upgrading. HBase writes incoming data to an in-memory store, called a _memstore_. After the memstore reaches a certain size, HBase flushes it to disk for long-term storage in the cluster's storage account. When deleting the old cluster, the memstores are recycled, potentially losing data. To manually flush the memstore for each table to disk, run the following script. The latest version of this script is on Azure's [GitHub](https://raw.githubusercontent.com/Azure/hbase-utils/master/scripts/flush_all_tables.sh).
57
53
58
54
```bash
59
55
#!/bin/bash
@@ -170,41 +166,47 @@ To upgrade your Apache HBase cluster on Azure HDInsight, complete the following
170
166
...
171
167
172
168
```
173
-
174
-
4. Stop ingestion to the old HBase cluster.
175
-
5. To ensure that any recent data in the memstore is flushed, run the previous script again.
176
-
6. Sign in to [Apache Ambari](https://ambari.apache.org/) on the old cluster (https://OLDCLUSTERNAME.azurehdidnsight.net) and stop the HBase services. When you prompted to confirm that you'd like to stop the services, check the box to turn on maintenance mode for HBase. For more information on connecting to and using Ambari, see [Manage HDInsight clusters by using the Ambari Web UI](../hdinsight-hadoop-manage-ambari.md).
169
+
170
+
1. Stop ingestion to the old HBase cluster.
171
+
172
+
1. To ensure that any recent data in the memstore is flushed, run the previous script again.
173
+
174
+
1. Sign in to [Apache Ambari](https://ambari.apache.org/) on the old cluster (`https://OLDCLUSTERNAME.azurehdidnsight.net`) and stop the HBase services. When you prompted to confirm that you'd like to stop the services, check the box to turn on maintenance mode for HBase. For more information on connecting to and using Ambari, see [Manage HDInsight clusters by using the Ambari Web UI](../hdinsight-hadoop-manage-ambari.md).
177
175
178
176

179
177
180
178

181
179
182
-
7. Sign in to Ambari on the new HDInsight cluster. Change the `fs.defaultFS` HDFS setting to point to the container name used by the original cluster. This setting is under **HDFS > Configs > Advanced > Advanced core-site**.
180
+
1. Sign in to Ambari on the new HDInsight cluster. Change the `fs.defaultFS` HDFS setting to point to the container name used by the original cluster. This setting is under **HDFS > Configs > Advanced > Advanced core-site**.

187
185
188
-
8. **If you aren't using HBase clusters with the Enhanced Writes feature, skip this step. It's needed only for HBase clusters with Enhanced Writes feature.**
189
-
186
+
1. If you aren't using HBase clusters with the Enhanced Writes feature, skip this step. It's needed only for HBase clusters with Enhanced Writes feature.
187
+
190
188
Change the `hbase.rootdir` path to point to the container of the original cluster.
191
189
192
190

191
+
193
192
1. If you're upgrading HDInsight 3.6 to 4.0, follow the steps below, otherwise skip to step 10:
194
193
1. Restart all required services in Ambari by selecting **Services**>**Restart All Required**.
195
194
1. Stop the HBase service.
196
195
1. SSH to the Zookeeper node, and execute the [zkCli](https://github.com/go-zkcli/zkcli) command`rmr /hbase-unsecure` to remove the HBase root znode from Zookeeper.
197
196
1. Restart HBase.
197
+
198
198
1. If you're upgrading to any other HDInsight version besides 4.0, follow these steps:
199
199
1. Save your changes.
200
200
1. Restart all required services as indicated by Ambari.
201
+
201
202
1. Point your application to the new cluster.
202
203
203
204
> [!NOTE]
204
205
> The static DNS for your application changes when upgrading. Rather than hard-coding this DNS, you can configure a CNAME in your domain name's DNS settings that points to the cluster's name. Another option is to use a configuration file for your application that you can update without redeploying.
205
206
206
-
12. Start the ingestion to see if everything is functioning as expected.
207
-
13. If the new cluster is satisfactory, delete the original cluster.
207
+
1. Start the ingestion to see if everything is functioning as expected.
208
+
209
+
1. If the new cluster is satisfactory, delete the original cluster.
0 commit comments