You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<sup>1</sup> Requests that clients make by using NFS 3.0 or SFTP can't be authorized by using Microsoft Entra security.
81
81
82
82
<sup>2</sup> Only locally redundant storage (LRS) and zone-redundant storage (ZRS) are supported.
83
83
84
-
<sup>3</sup> Storage Analytics metrics is retired. See [Transition to metrics in Azure Monitor](../common/storage-analytics-metrics.md?toc=/azure/storage/blobs/toc.json).
85
-
86
84
## Premium block blob accounts
87
85
88
86
The following table describes whether a feature is supported in a premium block blob account when you enable a hierarchical namespace (HNS), NFS 3.0 protocol, or SFTP.
@@ -125,14 +123,12 @@ The following table describes whether a feature is supported in a premium block
125
123
|[Soft delete for containers](soft-delete-container-overview.md)|✅|✅|✅|✅|
<sup>1</sup> Requests that clients make by using NFS 3.0 or SFTP can't be authorized by using Microsoft Entra security.
131
129
132
130
<sup>2</sup> Only locally redundant storage (LRS) and zone-redundant storage (ZRS) are supported.
133
131
134
-
<sup>3</sup> Storage Analytics metrics is retired. See [Transition to metrics in Azure Monitor](../common/storage-analytics-metrics.md?toc=/azure/storage/blobs/toc.json).
135
-
136
132
## See also
137
133
138
134
-[Known issues with Azure Data Lake Storage Gen2](data-lake-storage-known-issues.md)
Copy file name to clipboardExpand all lines: articles/storage/common/storage-account-upgrade.md
+44-19Lines changed: 44 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ author: akashdubey-ms
8
8
ms.service: azure-storage
9
9
ms.subservice: storage-common-concepts
10
10
ms.topic: how-to
11
-
ms.date: 01/09/2023
11
+
ms.date: 08/17/2023
12
12
ms.author: akashdubey
13
13
ms.custom: devx-track-azurecli, engagement
14
14
---
@@ -111,34 +111,59 @@ To estimate the cost of storing and accessing blob data in a general-purpose v2
111
111
112
112
To decide on the best access tier for your needs, it can be helpful to determine your blob data capacity, and how that data is being used. This can be best done by looking at the monitoring metrics for your account.
113
113
114
-
To estimate the cost of storing and accessing blob data in a general-purpose v2 storage account in a particular tier, evaluate your existing usage pattern or approximate your expected usage pattern. In general, you want to know:
114
+
### Monitoring existing storage accounts
115
115
116
-
- Your Blob storage consumption, in gigabytes, including:
117
-
- How much data is being stored in the storage account?
118
-
- How does the data volume change on a monthly basis; does new data constantly replace old data?
116
+
To monitor your existing storage accounts and gather this data, you can make use of Azure Storage Analytics, which performs logging and provides metrics data for a storage account. Storage Analytics can store metrics that include aggregated transaction statistics and capacity data about requests to the storage service for GPv1, GPv2, and Blob storage account types. This data is stored in well-known tables in the same storage account.
119
117
120
-
- The primary access pattern for your Blob storage data, including:
121
-
- How much data is being read from and written to the storage account?
122
-
- How many read operations versus write operations occur on the data in the storage account?
118
+
For more information, see [About Storage Analytics Metrics](../blobs/monitor-blob-storage.md) and [Storage Analytics Metrics Table Schema](/rest/api/storageservices/Storage-Analytics-Metrics-Table-Schema)
123
119
124
-
To decide on the best access tier for your needs, it can be helpful to determine your blob data capacity, and how that data is being used. This can be best done by looking at the monitoring metrics for your account.
120
+
> [!NOTE]
121
+
> Blob storage accounts expose the Table service endpoint only for storing and accessing the metrics data for that account.
125
122
126
-
### Monitoring existing storage accounts
123
+
To monitor the storage consumption for Blob storage, you need to enable the capacity metrics.
124
+
With this enabled, capacity data is recorded daily for a storage account's Blob service and recorded as a table entry that is written to the *$MetricsCapacityBlob* table within the same storage account.
127
125
128
-
To monitor your existing storage accounts and gather this data, you can make use of storage metrics in Azure Monitor. Azure Monitor stores metrics that include aggregated transaction statistics and capacity data about requests to the storage service. Azure Storage sends metric data to the Azure Monitor back end. Azure Monitor provides a unified monitoring experience that includes data from the Azure portal as well as data that is ingested. For more information, see any of these articles:
126
+
To monitor data access patterns for Blob storage, you need to enable the hourly transaction metrics from the API. With hourly transaction metrics enabled, per API transactions are aggregated every hour, and recorded as a table entry that is written to the *$MetricsHourPrimaryTransactionsBlob* table within the same storage account. The *$MetricsHourSecondaryTransactionsBlob* table records the transactions to the secondary endpoint when using RA-GRS storage accounts.
> If you have a general-purpose storage account in which you have stored page blobs and virtual machine disks, or queues, files, or tables, alongside block and append blob data, this estimation process isn't applicable. The capacity data doesn't differentiate block blobs from other types, and doesn't give capacity data for other data types. If you use these types, an alternative methodology is to look at the quantities on your most recent bill.
134
130
135
-
In order to estimate the data access costs for Blob storage accounts, you need to break down the transactions into two groups.
131
+
To get a good approximation of your data consumption and access pattern, we recommend you choose a retention period for the metrics that is representative of your regular usage and extrapolate. One option is to retain the metrics data for seven days and collect the data every week, for analysis at the end of the month. Another option is to retain the metrics data for the last 30 days and collect and analyze the data at the end of the 30-day period.
132
+
133
+
For details on enabling, collecting, and viewing metrics data, see [Storage analytics metrics](../common/storage-analytics-metrics.md?toc=/azure/storage/blobs/toc.json).
136
134
137
-
- The amount of data retrieved from the storage account can be estimated by looking at the sum of the *'Ingress'* metric for primarily the *'GetBlob'* and *'CopyBlob'* operations.
135
+
> [!NOTE]
136
+
> Storing, accessing, and downloading analytics data is also charged just like regular user data.
137
+
138
+
### Utilizing usage metrics to estimate costs
139
+
140
+
#### Capacity costs
141
+
142
+
The latest entry in the capacity metrics table *$MetricsCapacityBlob* with the row key *'data'* shows the storage capacity consumed by user data. The latest entry in the capacity metrics table *$MetricsCapacityBlob* with the row key *'analytics'* shows the storage capacity consumed by the analytics logs.
143
+
144
+
This total capacity consumed by both user data and analytics logs (if enabled) can then be used to estimate the cost of storing data in the storage account. The same method can also be used for estimating storage costs in GPv1 storage accounts.
145
+
146
+
#### Transaction costs
147
+
148
+
The sum of *'TotalBillableRequests'*, across all entries for an API in the transaction metrics table indicates the total number of transactions for that particular API. *For example*, the total number of *'GetBlob'* transactions in a given period can be calculated by the sum of total billable requests for all entries with the row key *'user;GetBlob'*.
149
+
150
+
In order to estimate transaction costs for Blob storage accounts, you need to break down the transactions into three groups since they're priced differently.
151
+
152
+
- Write transactions such as *'PutBlob'*, *'PutBlock'*, *'PutBlockList'*, *'AppendBlock'*, *'ListBlobs'*, *'ListContainers'*, *'CreateContainer'*, *'SnapshotBlob'*, and *'CopyBlob'*.
153
+
- Delete transactions such as *'DeleteBlob'* and *'DeleteContainer'*.
154
+
- All other transactions.
155
+
156
+
In order to estimate transaction costs for GPv1 storage accounts, you need to aggregate all transactions irrespective of the operation/API.
157
+
158
+
#### Data access and geo-replication data transfer costs
159
+
160
+
While storage analytics doesn't provide the amount of data read from and written to a storage account, it can be roughly estimated by looking at the transaction metrics table. The sum of *'TotalIngress'* across all entries for an API in the transaction metrics table indicates the total amount of ingress data in bytes for that particular API. Similarly the sum of *'TotalEgress'* indicates the total amount of egress data, in bytes.
161
+
162
+
In order to estimate the data access costs for Blob storage accounts, you need to break down the transactions into two groups.
138
163
139
-
- The amount of data written to the storage account can be estimated by looking at the sum of *'Egress'*metrics for primarily the *'PutBlob'*, *'PutBlock'*, *'CopyBlob'*and *'AppendBlock'* operations.
164
+
- The amount of data retrieved from the storage account can be estimated by looking at the sum of *'TotalEgress'* for primarily the *'GetBlob'*and *'CopyBlob'* operations.
140
165
141
-
To determine the price of each operation against the blob storage service, see [Map each REST operation to a price](../blobs/map-rest-apis-transaction-categories.md).
166
+
- The amount of data written to the storage account can be estimated by looking at the sum of *'TotalIngress'* for primarily the *'PutBlob'*, *'PutBlock'*, *'CopyBlob'* and *'AppendBlock'* operations.
142
167
143
168
The cost of geo-replication data transfer for Blob storage accounts can also be calculated by using the estimate for the amount of data written when using a GRS or RA-GRS storage account.
title: Use Azure Storage analytics to collect log data
2
+
title: Use Azure Storage analytics to collect logs and metrics data
3
3
description: Storage Analytics enables you to track metrics data for all storage services, and to collect logs for Blob, Queue, and Table storage.
4
4
author: normesta
5
5
ms.service: azure-storage
6
6
ms.topic: conceptual
7
-
ms.date: 01/09/2023
7
+
ms.date: 03/03/2017
8
8
ms.author: normesta
9
9
ms.reviewer: fryu
10
10
ms.subservice: storage-common-concepts
@@ -13,37 +13,35 @@ ms.custom: monitoring
13
13
14
14
# Storage Analytics
15
15
16
-
Azure Storage Analytics performs logging for a storage account. You can use this data to trace requests, analyze usage trends, and diagnose issues with your storage account.
17
-
18
-
> [!NOTE]
19
-
> Storage Analytics supports only logs. Storage Analytics metrics are retired. See [Transition to metrics in Azure Monitor](../common/storage-analytics-metrics.md?toc=/azure/storage/blobs/toc.json). While Storage Analytics logs are still suppported, we recommend that you use Azure Storage logs in Azure Monitor instead of Storage Analytics logs. To learn more, see any of the following articles:
Azure Storage Analytics performs logging and provides metrics data for a storage account. You can use this data to trace requests, analyze usage trends, and diagnose issues with your storage account.
25
17
26
18
To use Storage Analytics, you must enable it individually for each service you want to monitor. You can enable it from the [Azure portal](https://portal.azure.com). For details, see [Monitor a storage account in the Azure portal](./manage-storage-analytics-logs.md). You can also enable Storage Analytics programmatically via the REST API or the client library. Use the [Set Blob Service Properties](/rest/api/storageservices/set-blob-service-properties), [Set Queue Service Properties](/rest/api/storageservices/set-queue-service-properties), [Set Table Service Properties](/rest/api/storageservices/set-table-service-properties), and [Set File Service Properties](/rest/api/storageservices/Get-File-Service-Properties) operations to enable Storage Analytics for each service.
27
19
28
-
The aggregated log data is stored in a well-known blob, which may be accessed using the Blob service and Table service APIs.
20
+
The aggregated data is stored in a well-known blob (for logging) and in well-known tables (for metrics), which may be accessed using the Blob service and Table service APIs.
29
21
30
22
Storage Analytics has a 20 TB limit on the amount of stored data that is independent of the total limit for your storage account. For more information about storage account limits, see [Scalability and performance targets for standard storage accounts](scalability-targets-standard-account.md).
31
23
32
24
For an in-depth guide on using Storage Analytics and other tools to identify, diagnose, and troubleshoot Azure Storage-related issues, see [Monitor, diagnose, and troubleshoot Microsoft Azure Storage](storage-monitoring-diagnosing-troubleshooting.md).
33
25
34
26
## Billing for Storage Analytics
35
27
36
-
The amount of storage used by logs data is billable. You're also billed for requests to create blobs for logging.
28
+
All metrics data is written by the services of a storage account. As a result, each write operation performed by Storage Analytics is billable. Additionally, the amount of storage used by metrics data is also billable.
29
+
30
+
The following actions performed by Storage Analytics are billable:
31
+
32
+
- Requests to create blobs for logging.
33
+
- Requests to create table entities for metrics.
37
34
38
-
If you have configured a data retention policy, you can reduce the spending by deleting old log data. For more information about retention policies, see [Setting a Storage Analytics Data Retention Policy](/rest/api/storageservices/Setting-a-Storage-Analytics-Data-Retention-Policy).
35
+
If you have configured a data retention policy, you can reduce the spending by deleting old logging and metrics data. For more information about retention policies, see [Setting a Storage Analytics Data Retention Policy](/rest/api/storageservices/Setting-a-Storage-Analytics-Data-Retention-Policy).
39
36
40
37
### Understanding billable requests
41
38
42
-
Every request made to an account's storage service is either billable or non-billable. Storage Analytics logs each individual request made to a service, including a status message that indicates how the request was handled. See[Understanding Azure Storage Billing - Bandwidth, Transactions, and Capacity](/archive/blogs/windowsazurestorage/understanding-windows-azure-storage-billing-bandwidth-transactions-and-capacity).
39
+
Every request made to an account's storage service is either billable or non-billable. Storage Analytics logs each individual request made to a service, including a status message that indicates how the request was handled. Similarly, Storage Analytics stores metrics for both a service and the API operations of that service, including the percentages and count of certain status messages. Together, these features can help you analyze your billable requests, make improvements on your application, and diagnose issues with requests to your services. For more information about billing, see[Understanding Azure Storage Billing - Bandwidth, Transactions, and Capacity](/archive/blogs/windowsazurestorage/understanding-windows-azure-storage-billing-bandwidth-transactions-and-capacity).
43
40
44
-
When looking at Storage Analytics data, you can use the tables in the [Storage Analytics Logged Operations and Status Messages](/rest/api/storageservices/storage-analytics-logged-operations-and-status-messages) topic to determine what requests are billable. Then you can compare your log data to the status messages to see if you were charged for a particular request. You can also use the tables in the previous topic to investigate availability for a storage service or individual API operation.
41
+
When looking at Storage Analytics data, you can use the tables in the [Storage Analytics Logged Operations and Status Messages](/rest/api/storageservices/storage-analytics-logged-operations-and-status-messages) topic to determine what requests are billable. Then you can compare your logs and metrics data to the status messages to see if you were charged for a particular request. You can also use the tables in the previous topic to investigate availability for a storage service or individual API operation.
45
42
46
43
## Next steps
47
44
48
45
-[Monitor a storage account in the Azure portal](./manage-storage-analytics-logs.md)
Copy file name to clipboardExpand all lines: articles/storage/common/storage-metrics-migration.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ description: Learn how to transition from Storage Analytics metrics (classic met
4
4
author: normesta
5
5
ms.service: azure-storage
6
6
ms.topic: conceptual
7
-
ms.date: 01/09/2024
7
+
ms.date: 01/03/2024
8
8
ms.author: normesta
9
9
ms.reviewer: fryu
10
10
ms.subservice: storage-common-concepts
@@ -13,7 +13,7 @@ ms.custom: monitoring
13
13
14
14
# Transition to metrics in Azure Monitor
15
15
16
-
On **January 9, 2024** Storage Analytics metrics, also referred to as *classic metrics* retired. If you used classic metrics, this article helps you transition to metrics in Azure Monitor.
16
+
On **January 9, 2024** Storage Analytics metrics, also referred to as *classic metrics*will be retired. If you use classic metrics, make sure to transition to metrics in Azure Monitor prior to that date. This article helps you make the transition.
0 commit comments