Skip to content

Commit 3a3d000

Browse files
authored
Update hdinsight-50-component-versioning.md
1 parent 68aeaef commit 3a3d000

File tree

1 file changed

+13
-8
lines changed

1 file changed

+13
-8
lines changed

articles/hdinsight/hdinsight-50-component-versioning.md

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,10 @@ The Open-source component versions associated with HDInsight 5.0 are listed in t
1818

1919
| Component | HDInsight 5.0 |
2020
|------------------------|---------------|
21-
|Apache Spark | 3.1 |
22-
|Apache Interactive Query |3.1 |
21+
|Apache Spark | 3.1.2 |
2322
|Apache Hive | 3.1.2 |
24-
|Apache Kafka | 2.4 |
25-
|Apache Hadoop and YARN |3.1.1 |
23+
|Apache Kafka | 2.4.1 |
24+
|Apache Hadoop |3.1.1 |
2625
|Apache Tez | 0.9.1 |
2726
|Apache Pig | 0.16.0 |
2827
|Apache Ranger | 1.1.0 |
@@ -37,8 +36,6 @@ This table lists certain HDInsight 4.0 cluster types that have retired or will b
3736

3837
| Cluster Type | Framework version | Support expiration date | Retirement date |
3938
|---------------------------------|-------------------|------------------------------|-----------------|
40-
| HDInsight 4.0 Spark | 2.3 | June 30, 2020 | June 30, 2020 |
41-
| HDInsight 4.0 Kafka | 1.1 | Dec 31, 2020 | Dec 31, 2020 |
4239
| HDInsight 4.0 Kafka | 2.1.0 | Sep 30, 2022 | Oct 1, 2022 |
4340

4441
## Spark
@@ -47,15 +44,17 @@ This table lists certain HDInsight 4.0 cluster types that have retired or will b
4744

4845
> [!NOTE]
4946
> * If you are using Azure User Interface to create a Spark Cluster for HDInsight, you will see from the dropdown list an additional version Spark 3.1.(HDI 5.0) along with the older versions. This version is a renamed version of Spark 3.1.(HDI 4.0) and it is backward compatible.
50-
> * This is only an UI level change, which doesn’t impact anything for the existing users and users who are already using the ARM template to build their clusters. For backward compatibility, ARM supports creating spark 4.0 and 5.0 for 3.1 version which maps to same versions spark 3.1 (5.0)
47+
> * This is only an UI level change, which doesn’t impact anything for the existing users and users who are already using the ARM template to build their clusters. > * For backward compatibility, ARM supports creating Spark 3.1 with HDI 4.0 and 5.0 versions which maps to same versions Sspark 3.1 (HDI 5.0)
48+
> * Spark 3.1 (HDI 5.0) cluster comes with HWC 2.0 which works well together with Interactive Query (HDI 5.0) cluster.
5149
5250
## Interactive Query
5351

5452
:::image type="content" source="./media/hdinsight-release-notes/interactive-query-3-1-for-hdi-5-0.png" alt-text="Screenshot of interactive query 3.1 for HDI 5.0":::
5553

5654
> [!NOTE]
5755
> * If you are creating an Interactive Query Cluster, you will see from the dropdown list an other version as Interactive Query 3.1 (HDI 5.0).
58-
> * If you are going to use Spark 3.1 version along with Hive which require ACID support, you need to select this version Interactive Query 3.1 (HDI 5.0).
56+
> * If you are going to use Spark 3.1 version along with Hive which require ACID support via Hive Warehouse Connector (HWC).
57+
you need to select this version Interactive Query 3.1 (HDI 5.0).
5958

6059
## Kafka
6160

@@ -65,6 +64,12 @@ This table lists certain HDInsight 4.0 cluster types that have retired or will b
6564

6665
We are working on this issue, and a fix will be rolled out shortly.
6766

67+
### Upcoming version upgrades.
68+
HDInsight team is working on upgrading other open-source components.
69+
1. Spark 3.2.0
70+
1. Kafka 3.2.1
71+
1. HBase 2.4.9
72+
6873

6974
## Next steps
7075

0 commit comments

Comments
 (0)