You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/storage/files/storage-files-scale-targets.md
+28-24Lines changed: 28 additions & 24 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ description: Learn about the capacity, IOPS, and throughput rates for Azure file
4
4
author: khdownie
5
5
ms.service: azure-file-storage
6
6
ms.topic: conceptual
7
-
ms.date: 01/25/2024
7
+
ms.date: 03/22/2024
8
8
ms.author: kendownie
9
9
---
10
10
@@ -28,7 +28,7 @@ Azure file shares are deployed into storage accounts, which are top-level object
28
28
29
29
### Storage account scale targets
30
30
31
-
Storage account scale targets apply at the storage account level. There are two main types of storage accounts for Azure Files:
31
+
Storage account scale targets apply at the storage account level. There are two main types of storage accounts for Azure Files:
32
32
33
33
-**General purpose version 2 (GPv2) storage accounts**: GPv2 storage accounts allow you to deploy Azure file shares on standard/hard disk-based (HDD-based) hardware. In addition to storing Azure file shares, GPv2 storage accounts can store other storage resources such as blob containers, queues, or tables. File shares can be deployed into the transaction optimized (default), hot, or cool tiers.
34
34
@@ -73,7 +73,7 @@ Azure file share scale targets apply at the file share level.
73
73
74
74
<sup>1</sup> The limits for standard file shares apply to all three of the tiers available for standard file shares: transaction optimized, hot, and cool.
75
75
76
-
<sup>2</sup> Default on standard file shares is 5 TiB, see [Create an Azure file share](./storage-how-to-create-file-share.md) for the details on how to create file shares with 100 TiB size and increase existing standard file shares up to 100 TiB. To take advantage of the larger scale targets, you must change your quota so that it is larger than 5 TiB.
76
+
<sup>2</sup> Default on standard file shares is 5 TiB, see [Create an Azure file share](./storage-how-to-create-file-share.md) for the details on how to create file shares with 100 TiB size and increase existing standard file shares up to 100 TiB. To take advantage of the larger scale targets, you must change your quota so that it's larger than 5 TiB.
77
77
78
78
<sup>3</sup> Azure Files enforces certain [naming rules](/rest/api/storageservices/naming-and-referencing-shares--directories--files--and-metadata#directory-and-file-names) for directory and file names.
79
79
@@ -138,6 +138,7 @@ The following table indicates which targets are soft, representing the Microsoft
138
138
| Storage Sync Services per region | 100 Storage Sync Services | Yes |
139
139
| Sync groups per Storage Sync Service | 200 sync groups | Yes |
140
140
| Registered servers per Storage Sync Service | 99 servers | Yes |
141
+
| Private endpoints per Storage Sync Service | 100 private endpoints | Yes |
141
142
| Cloud endpoints per sync group | 1 cloud endpoint | Yes |
142
143
| Server endpoints per sync group | 100 server endpoints | Yes |
143
144
| Server endpoints per server | 30 server endpoints | Yes |
@@ -152,18 +153,19 @@ The following table indicates which targets are soft, representing the Microsoft
152
153
153
154
## Azure File Sync performance metrics
154
155
155
-
Since the Azure File Sync agent runs on a Windows Server machine that connects to the Azure file shares, the effective sync performance depends upon a number of factors in your infrastructure: Windows Server and the underlying disk configuration, network bandwidth between the server and the Azure storage, file size, total dataset size, and the activity on the dataset. Since Azure File Sync works on the file level, the performance characteristics of an Azure File Sync-based solution should be measured by the number of objects (files and directories) processed per second.
156
+
Since the Azure File Sync agent runs on a Windows Server machine that connects to the Azure file shares, effective sync performance depends upon a number of factors in your infrastructure: Windows Server and the underlying disk configuration, network bandwidth between the server and the Azure storage, file size, total dataset size, and the activity on the dataset. Since Azure File Sync works on the file level, the performance characteristics of an Azure File Sync-based solution should be measured by the number of objects (files and directories) processed per second.
156
157
157
158
For Azure File Sync, performance is critical in two stages:
158
159
159
160
1.**Initial one-time provisioning**: To optimize performance on initial provisioning, refer to [Onboarding with Azure File Sync](../file-sync/file-sync-deployment-guide.md#onboarding-with-azure-file-sync) for the optimal deployment details.
160
161
2.**Ongoing sync**: After the data is initially seeded in the Azure file shares, Azure File Sync keeps multiple endpoints in sync.
161
-
> [!Note]
162
-
> When many server endpoints in the same sync group are syncing at the same time, they are contending for cloud service resources. As a result, upload performance will be impacted. In extreme cases, some sync sessions will fail to access the resources, and will fail. However, those sync sessions will resume shortly and eventually succeed once the congestion is reduced.
162
+
163
+
> [!NOTE]
164
+
> When many server endpoints in the same sync group are syncing at the same time, they're contending for cloud service resources. As a result, upload performance is impacted. In extreme cases, some sync sessions will fail to access the resources, and will fail. However, those sync sessions will resume shortly and eventually succeed once the congestion is reduced.
163
165
164
166
## Internal test results
165
167
166
-
To help you plan your deployment for each of the stages (initial one-time provisioning and ongoing sync), below are the results observed during the internal testing on a system with the following configuration:
168
+
To help you plan your deployment for each of the stages (initial one-time provisioning and ongoing sync), here are the results we observed during internal testing on a system with the following configuration:
167
169
168
170
| System configuration | Details |
169
171
|-|-|
@@ -178,50 +180,52 @@ To help you plan your deployment for each of the stages (initial one-time provis
| Initial cloud change enumeration | 80 objects per second |
184
186
| Upload Throughput | 20 objects per second per sync group |
185
187
| Namespace Download Throughput | 400 objects per second |
186
188
187
-
**Initial cloud change enumeration**: When a new sync group is created, initial cloud change enumeration is the first step that will execute. In this process, the system will enumerate all the items in the Azure File Share. During this process, there will be no sync activity i.e. no items will be downloaded from cloud endpoint to server endpoint and no items will be uploaded from server endpoint to cloud endpoint. Sync activity will resume once initial cloud change enumeration completes.
188
-
The rate of performance is 80 objects per second. Customers can estimate the time it will take to complete initial cloud change enumeration by determining the number of items in the cloud share and using the following formulae to get the time in days.
189
+
**Initial cloud change enumeration**: When a new sync group is created, initial cloud change enumeration is the first step that executes. In this process, the system will enumerate all the items in the Azure file share. During this process, there will be no sync activity. No items will be downloaded from cloud endpoint to server endpoint, and no items will be uploaded from server endpoint to cloud endpoint. Sync activity will resume once initial cloud change enumeration completes.
190
+
191
+
The rate of performance is 80 objects per second. You can estimate the time it will take to complete initial cloud change enumeration by determining the number of items in the cloud share and using the following formulae to get the time in days.
189
192
190
-
**Time (in days) for initial cloud enumeration = (Number of objects in cloud endpoint)/(80 * 60 * 60 * 24)**
193
+
**Time (in days) for initial cloud enumeration = (Number of objects in cloud endpoint)/(80 \* 60 \* 60 \* 24)**
191
194
192
-
**Initial sync of data from Windows Server to Azure File share**:Many Azure File Sync deployments start with an empty Azure file share because all the data is on the Windows Server. In these cases, the initial cloud change enumeration is fast and the majority of time will be spent syncing changes from the Windows Server into the Azure file share(s).
195
+
**Initial sync of data from Windows Server to Azure File share:**Many Azure File Sync deployments start with an empty Azure file share because all the data is on the Windows Server. In these cases, the initial cloud change enumeration is fast, and the majority of time is spent syncing changes from the Windows Server into the Azure file share(s).
193
196
194
-
While sync uploads data to the Azure file share, there is no downtime on the local file server, and administrators can [setup network limits](../file-sync/file-sync-server-registration.md#set-azure-file-sync-network-limits) to restrict the amount of bandwidth used for background data upload.
197
+
While sync uploads data to the Azure file share, there's no downtime on the local file server, and administrators can [setup network limits](../file-sync/file-sync-server-registration.md#set-azure-file-sync-network-limits) to restrict the amount of bandwidth used for background data upload.
195
198
196
-
Initial sync is typically limited by the initial upload rate of 20 files per second per sync group. Customers can estimate the time to upload all their data to Azure using the following formulae to get time in days:
199
+
Initial sync is typically limited by the initial upload rate of 20 files per second per sync group. Customers can estimate the time to upload all their data to Azure using the following formulae to get time in days:
197
200
198
-
**Time (in days) for uploading files to a sync group = (Number of objects in server endpoint)/(20 * 60 * 60 * 24)**
201
+
**Time (in days) for uploading files to a sync group = (Number of objects in server endpoint)/(20 \* 60 \* 60 \* 24)**
199
202
200
203
Splitting your data into multiple server endpoints and sync groups can speed up this initial data upload, because the upload can be done in parallel for multiple sync groups at a rate of 20 items per second each. So, two sync groups would be running at a combined rate of 40 items per second. The total time to complete would be the time estimate for the sync group with the most files to sync.
201
204
202
-
**Namespace download throughput** When a new server endpoint is added to an existing sync group, the Azure File Sync agent does not download any of the file content from the cloud endpoint. It first syncs the full namespace and then triggers background recall to download the files, either in their entirety or, if cloud tiering is enabled, to the cloud tiering policy set on the server endpoint.
205
+
**Namespace download throughput:** When a new server endpoint is added to an existing sync group, the Azure File Sync agent doesn't download any of the file content from the cloud endpoint. It first syncs the full namespace and then triggers background recall to download the files, either in their entirety or, if cloud tiering is enabled, to the cloud tiering policy set on the server endpoint.
203
206
204
207
### Ongoing sync
205
208
206
209
| Ongoing sync | Details |
207
210
|-|--|
208
-
| Number of objects synced| 125,000 objects (~1% churn) |
209
-
| Dataset Size| 50 GiB |
211
+
| Number of objects synced| 125,000 objects (~1% churn) |
212
+
| Dataset Size| 50 GiB |
210
213
| Average File Size |~500 KiB |
211
214
| Upload Throughput | 20 objects per second per sync group |
212
-
| Full Download Throughput*| 60 objects per second |
215
+
| Full Download Throughput\*| 60 objects per second |
213
216
214
-
*If cloud tiering is enabled, you are likely to observe better performance as only some of the file data is downloaded. Azure File Sync only downloads the data of cached files when they are changed on any of the endpoints. For any tiered or newly created files, the agent does not download the file data, and instead only syncs the namespace to all the server endpoints. The agent also supports partial downloads of tiered files as they are accessed by the user.
217
+
\*If cloud tiering is enabled, you're likely to observe better performance as only some of the file data is downloaded. Azure File Sync only downloads the data of cached files when they're changed on any of the endpoints. For any tiered or newly created files, the agent doesn't download the file data, and instead only syncs the namespace to all the server endpoints. The agent also supports partial downloads of tiered files as they're accessed by the user.
215
218
216
-
> [!Note]
217
-
> The numbers above are not an indication of the performance that you will experience. The actual performance will depend on multiple factors as outlined in the beginning of this section.
219
+
> [!NOTE]
220
+
> These numbers aren't an indication of the performance that you'll experience. The actual performance depends on multiple factors as outlined in the beginning of this section.
218
221
219
-
As a general guide for your deployment, you should keep a few things in mind:
222
+
As a general guide for your deployment, keep a few things in mind:
220
223
221
224
- The object throughput approximately scales in proportion to the number of sync groups on the server. Splitting data into multiple sync groups on a server yields better throughput, which is also limited by the server and network.
222
-
- The object throughput is inversely proportional to the MiB per second throughput. For smaller files, you will experience higher throughput in terms of the number of objects processed per second, but lower MiB per second throughput. Conversely, for larger files, you will get fewer objects processed per second, but higher MiB per second throughput. The MiB per second throughput is limited by the Azure Files scale targets.
225
+
- The object throughput is inversely proportional to the MiB per second throughput. For smaller files, you'll experience higher throughput in terms of the number of objects processed per second, but lower MiB per second throughput. Conversely, for larger files, you'll get fewer objects processed per second, but higher MiB per second throughput. The MiB per second throughput is limited by the Azure Files scale targets.
0 commit comments