You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-compare-storage-options.md
+16-17Lines changed: 16 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,7 +3,7 @@ title: Compare storage options for use with Azure HDInsight clusters
3
3
description: Provides an overview of storage types and how they work with Azure HDInsight.
4
4
ms.service: azure-hdinsight
5
5
ms.topic: conceptual
6
-
ms.date: 06/14/2024
6
+
ms.date: 12/17/2024
7
7
---
8
8
9
9
# Compare storage options for use with Azure HDInsight clusters
@@ -21,13 +21,13 @@ The following table summarizes the Azure Storage services that are supported wit
21
21
22
22
| Storage service | Account type | Namespace Type | Supported services | Supported performance tiers | Supported access tiers | HDInsight Version | Cluster type |
23
23
|---|---|---|---|---|---|---|---|
24
-
|Azure Data Lake Storage Gen2| General-purpose V2 | Hierarchical (filesystem) | Blob | Standard | Hot, Cool, Archive |3.6+| All except Spark 2.1 and 2.2|
25
-
|Azure Storage| General-purpose V2 | Object | Blob | Standard | Hot, Cool, Archive |3.6+| All |
24
+
|Azure Data Lake Storage Gen2| General-purpose V2 | Hierarchical (filesystem) | Blob | Standard | Hot, Cool, Archive |All| All except Spark 2.1 and 2.2|
25
+
|Azure Storage| General-purpose V2 | Object | Blob | Standard | Hot, Cool, Archive |All| All |
26
26
|Azure Storage| General-purpose V1 | Object | Blob | Standard | N/A | All | All |
27
27
|Azure Storage| Blob Storage**| Object | Block Blob | Standard | Hot, Cool, Archive | All | All |
28
-
|Azure Data Lake Storage Gen1| N/A | Hierarchical (filesystem) | N/A | N/A | N/A |3.6 Only| All except HBase |
29
-
|Azure Storage| Block Blob| Object | Block Blob | Premium | N/A|3.6+| Only HBase with accelerated writes|
30
-
|Azure Data Lake Storage Gen2| Block Blob| Hierarchical (filesystem) | Block Blob | Premium | N/A|3.6+ | Only HBase with accelerated writes|
28
+
|Azure Data Lake Storage Gen1| N/A | Hierarchical (filesystem) | N/A | N/A | N/A |All| All except HBase |
29
+
|Azure Storage| Block Blob| Object | Block Blob | Premium | N/A|All| Only HBase with accelerated writes|
30
+
|Azure Data Lake Storage Gen2| Block Blob| Hierarchical (filesystem) | Block Blob | Premium | N/A|All| Only HBase with accelerated writes|
31
31
32
32
**For HDInsight clusters, only secondary storage accounts can be of type BlobStorage and Page Blob isn't a supported storage option.
33
33
@@ -39,16 +39,15 @@ You can create clusters using combinations of services for primary and optional
| 3.6 & 4.0 | General Purpose V1, General Purpose V2 | General Purpose V1, General Purpose V2, BlobStorage(Block Blobs) | Yes |
43
-
| 3.6 & 4.0 | General Purpose V1, General Purpose V2 | Data Lake Storage Gen2 | No |
44
-
| 3.6 & 4.0 | Data Lake Storage Gen2*| Data Lake Storage Gen2 | Yes |
45
-
| 3.6 & 4.0 | Data Lake Storage Gen2*| General Purpose V1, General Purpose V2, BlobStorage(Block Blobs) | Yes |
46
-
| 3.6 & 4.0 | Data Lake Storage Gen2 | Data Lake Storage Gen1 | No |
47
-
| 3.6 | Data Lake Storage Gen1 | Data Lake Storage Gen1 | Yes |
48
-
| 3.6 | Data Lake Storage Gen1 | General Purpose V1, General Purpose V2, BlobStorage(Block Blobs) | Yes |
49
-
| 3.6 | Data Lake Storage Gen1 | Data Lake Storage Gen2 | No |
50
-
| 4.0 | Data Lake Storage Gen1 | Any | No |
51
-
| 4.0 | General Purpose V1, General Purpose V2 | Data Lake Storage Gen1 | No |
42
+
| 4.0, 5.0, 5.1 | General Purpose V1, General Purpose V2 | General Purpose V1, General Purpose V2, BlobStorage(Block Blobs) | Yes |
43
+
| 4.0, 5.0, 5.1 | General Purpose V1, General Purpose V2 | Data Lake Storage Gen2 | No |
44
+
| 4.0, 5.0, 5.1| Data Lake Storage Gen2 | Data Lake Storage Gen2 | Yes |
45
+
| 4.0, 5.0, 5.1 | Data Lake Storage Gen2*| General Purpose V1, General Purpose V2, BlobStorage(Block Blobs) | Yes |
46
+
| 4.0, 5.0, 5.1| Data Lake Storage Gen2 | Data Lake Storage Gen1 | No |
47
+
| 4.0, 5.0, 5.1 | Data Lake Storage Gen1 | Data Lake Storage Gen1 | Yes |
48
+
| 4.0, 5.0, 5.1 | Data Lake Storage Gen1 | General Purpose V1, General Purpose V2, BlobStorage(Block Blobs) | Yes |
49
+
| 4.0, 5.0, 5.1 | Data Lake Storage Gen1 | Data Lake Storage Gen2 | No |
50
+
| 4.0, 5.0, 5.1 | General Purpose V1, General Purpose V2 | Data Lake Storage Gen1 | No |
52
51
53
52
*=This could be one or multiple Data Lake Storage Gen2, as long as they're all setup to use the same managed identity for cluster access.
54
53
@@ -57,7 +56,7 @@ You can create clusters using combinations of services for primary and optional
57
56
58
57
## Data replication
59
58
60
-
Azure HDInsight does not store customer data. The primary means of storage for a cluster are its associated storage accounts. You can attach your cluster to an existing storage account, or create a new storage account during the cluster creation process. If a new account is created, it will be created as a locally redundant storage (LRS) account, and will satisfy in-region data residency requirements including those specified in the [Trust Center](https://azuredatacentermap.azurewebsites.net).
59
+
Azure HDInsight doesn't store customer data. The primary means of storage for a cluster are its associated storage accounts. You can attach your cluster to an existing storage account, or create a new storage account during the cluster creation process. If a new account is created, it will be created as a locally redundant storage (LRS) account, and will satisfy in-region data residency requirements including those specified in the [Trust Center](https://azuredatacentermap.azurewebsites.net).
61
60
62
61
You can validate that HDInsight is properly configured to store data in a single region by ensuring that the storage account associated with your HDInsight is LRS or another storage option mentioned on [Trust Center](https://azuredatacentermap.azurewebsites.net).
0 commit comments