Skip to content

Commit a2b61c4

Browse files
sreekzzsreekzz
authored
Updated Archived release notes
Updated Archived release notes
1 parent 427ffc7 commit a2b61c4

File tree

1 file changed

+105
-1
lines changed

1 file changed

+105
-1
lines changed

‎articles/hdinsight/hdinsight-release-notes-archive.md

Lines changed: 105 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Archived release notes for Azure HDInsight. Get development tips an
44
ms.service: hdinsight
55
ms.topic: conceptual
66
ms.custom: hdinsightactive, references_regions
7-
ms.date: 10/25/2022
7+
ms.date: 12/09/2022
88
---
99

1010
# Archived release notes
@@ -14,6 +14,110 @@ ms.date: 10/25/2022
1414
Azure HDInsight is one of the most popular services among enterprise customers for open-source analytics on Azure.
1515
If you would like to subscribe on release notes, watch releases on [this GitHub repository](https://github.com/hdinsight/release-notes/releases).
1616

17+
## Release date: 08/10/2022
18+
19+
This release applies to HDInsight 4.0.  HDInsight release is made available to all regions over several days.
20+
21+
HDInsight uses safe deployment practices, which involve gradual region deployment. It may take up to 10 business days for a new release or a new version to be available in all regions.
22+
23+
24+
![Icon_showing_new_features](media/hdinsight-release-notes/icon-for-new-feature.png)
25+
## New Feature
26+
27+
**1. Attach external disks in HDI Hadoop/Spark clusters**
28+
29+
HDInsight cluster comes with pre-defined disk space based on SKU. This space may not be sufficient in large job scenarios.
30+
31+
This new feature allows you to add more disks in cluster, which will be used as node manager local directory. Add number of disks to worker nodes during HIVE and Spark cluster creation, while the selected disks will be part of node manager’s local directories.
32+
33+
> [!NOTE]
34+
> The added disks are only configured for node manager local directories.
35+
>
36+
37+
For more information, [see here](./hdinsight-hadoop-provision-linux-clusters.md#configuration--pricing)
38+
39+
**2. Selective logging analysis**
40+
41+
Selective logging analysis is now available on all regions for public preview. You can connect your cluster to a log analytics workspace. Once enabled, you can see the logs and metrics like HDInsight Security Logs, Yarn Resource Manager, System Metrics etc. You can monitor workloads and see how they're affecting cluster stability. Selective logging allows you to enable/disable all the tables or enable selective tables in log analytics workspace. You can adjust the source type for each table, since in new version of Geneva monitoring one table has multiple sources.
42+
43+
1. The Geneva monitoring system uses mdsd(MDS daemon) which is a monitoring agent and fluentd for collecting logs using unified logging layer.
44+
1. Selective Logging uses script action to disable/enable tables and their log types. Since it doesn't open any new ports or change any existing security setting hence, there are no security changes.
45+
1. Script Action runs in parallel on all specified nodes and changes the configuration files for disabling/enabling tables and their log types.
46+
47+
For more information, [see here](./selective-logging-analysis.md)
48+
49+
50+
![Icon_showing_bug_fixes](media/hdinsight-release-notes/icon-for-bugfix.png)
51+
## Fixed
52+
53+
#### **Log analytics**
54+
55+
Log Analytics integrated with Azure HDInsight running OMS version 13 requires an upgrade to OMS version 14 to apply the latest security updates.
56+
Customers using older version of cluster with OMS version 13 need to install OMS version 14 to meet the security requirements. (How to check current version & Install 14)
57+
58+
**How to check your current OMS version**
59+
60+
1. Log in to the cluster using SSH.
61+
1. Run the following command in your SSH Client.
62+
63+
```
64+
sudo /opt/omi/bin/ominiserver/ --version
65+
```
66+
![Screenshot showing how to check OMS Upgrade](media/hdinsight-release-notes/check-oms-version.png)
67+
68+
**How to upgrade your OMS version from 13 to 14**
69+
70+
1. Log in to the [Azure portal](https://portal.azure.com/)
71+
1. From the resource group, select the HDInsight cluster resource
72+
1. Click **Script actions**
73+
1. From **Submit script action** panel, choose **Script type** as custom
74+
1. Paste the following link in the Bash script URL box
75+
https://hdiconfigactions.blob.core.windows.net/log-analytics-patch/OMSUPGRADE14.1/omsagent-vulnerability-fix-1.14.12-0.sh
76+
1. Select **Node type(s)**
77+
1. Click **Create**
78+
79+
![Screenshot showing how to do OMS Upgrade](media/hdinsight-release-notes/oms-upgrade.png)
80+
81+
1. Verify the successful installation of the patch using the following steps:
82+
83+
1. Log in to the cluster using SSH.
84+
1. Run the following command in your SSH Client.
85+
86+
```
87+
sudo /opt/omi/bin/ominiserver/ --version
88+
```
89+
90+
### Other bug fixes
91+
92+
1. Yarn log’s CLI failed to retrieve the logs if any TFile is corrupt or empty.
93+
2. Resolved invalid service principal details error while getting the OAuth token from Azure Active Directory.
94+
3. Improved cluster creation reliability when 100+ worked nodes are configured.
95+
96+
### Open source bug fixes
97+
98+
#### TEZ bug fixes
99+
100+
|Bug Fixes|Apache JIRA|
101+
|---|---|
102+
|Tez Build Failure: FileSaver.js not found|[TEZ-4411](https://issues.apache.org/jira/browse/TEZ-4411)|
103+
|Wrong FS Exception when warehouse and scratchdir are on different FS|[TEZ-4406](https://issues.apache.org/jira/browse/TEZ-4406)|
104+
|TezUtils.createConfFromByteString on Configuration larger than 32 MB throws com.google.protobuf.CodedInputStream exception|[TEZ-4142](https://issues.apache.org/jira/browse/TEZ-4142)|
105+
|TezUtils::createByteStringFromConf should use snappy instead of DeflaterOutputStream|[TEZ-4113](https://issues.apache.org/jira/browse/TEZ-4411)|
106+
|Update protobuf dependency to 3.x|[TEZ-4363](https://issues.apache.org/jira/browse/TEZ-4363)|
107+
108+
#### Hive bug fixes
109+
110+
|Bug Fixes|Apache JIRA|
111+
|---|---|
112+
|Perf optimizations in ORC split-generation| [HIVE-21457](https://issues.apache.org/jira/browse/HIVE-21457)|
113+
|Avoid reading table as ACID when table name is starting with "delta", but table isn't transactional and BI Split Strategy is used| [HIVE-22582](https://issues.apache.org/jira/browse/HIVE-22582)|
114+
|Remove an FS#exists call from AcidUtils#getLogicalLength|[HIVE-23533](https://issues.apache.org/jira/browse/HIVE-23533)|
115+
|Vectorized OrcAcidRowBatchReader.computeOffset and bucket optimization|[HIVE-17917](https://issues.apache.org/jira/browse/HIVE-17917)|
116+
117+
### Known issues
118+
119+
HDInsight is compatible with Apache HIVE 3.1.2. Due to a bug in this release, the Hive version is shown as 3.1.0 in hive interfaces. However, there's no impact on the functionality.
120+
17121
## Release date: 06/03/2022
18122

19123
This release applies for HDInsight 4.0. HDInsight release is made available to all regions over several days. The release date here indicates the first region release date. If you don't see below changes, wait for the release being live in your region over several days.

0 commit comments

Comments
 (0)