You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/storage/file-sync/file-sync-how-to-manage-tiered-files.md
+31-28Lines changed: 31 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,21 +4,21 @@ description: Tips and PowerShell commandlets to help you manage tiered files
4
4
author: khdownie
5
5
ms.service: azure-file-storage
6
6
ms.topic: how-to
7
-
ms.date: 06/06/2022
7
+
ms.date: 04/10/2024
8
8
ms.author: kendownie
9
9
---
10
10
11
11
# How to manage tiered files
12
12
13
-
This article provides guidance for users who have questions related to managing tiered files. For conceptual questions regarding cloud tiering, please see [Azure Files FAQ](../files/storage-files-faq.md?toc=/azure/storage/filesync/toc.json).
13
+
This article provides guidance for users who have questions related to managing tiered files. For conceptual questions regarding cloud tiering, see [Azure Files FAQ](../files/storage-files-faq.md?toc=/azure/storage/filesync/toc.json).
14
14
15
15
## How to check if your files are being tiered
16
16
17
17
Whether or not files need to be tiered per set policies is evaluated once an hour. You can come across two situations when a new server endpoint is created:
18
18
19
-
When you first add a new server endpoint, often files exist in that server location. They need to be uploaded before cloud tiering can begin. The volume free space policy will not begin its work until initial upload of all files has finished. However, the optional date policy will begin to work on an individual file basis, as soon as a file has been uploaded. The one-hour interval applies here as well.
19
+
1.When you first add a new server endpoint, often files exist in that server location. They need to be uploaded before cloud tiering can begin. The volume free space policy won't begin its work until initial upload of all files has finished. However, the optional date policy will begin to work on an individual file basis, as soon as a file has been uploaded. The one-hour interval applies here as well.
20
20
21
-
When you add a new server endpoint, it is possible you connected an empty server location to an Azure file share with your data in it. If you choose to download the namespace and recall content during initial download to your server, then after the namespace comes down, files will be recalled based on the last modified timestamp till the volume free space policy and the optional date policy limits are reached.
21
+
1.When you add a new server endpoint, it's possible you connected an empty server location to an Azure file share with your data in it. If you choose to download the namespace and recall content during initial download to your server, then after the namespace comes down, files will be recalled based on the last modified timestamp until the volume free space policy and the optional date policy limits are reached.
22
22
23
23
There are several ways to check whether a file has been tiered to your Azure file share:
24
24
@@ -29,9 +29,9 @@ There are several ways to check whether a file has been tiered to your Azure fil
29
29
|:----------------:|-----------|------------|
30
30
| A | Archive | Indicates that the file should be backed up by backup software. This attribute is always set, regardless of whether the file is tiered or stored fully on disk. |
31
31
| P | Sparse file | Indicates that the file is a sparse file. A sparse file is a specialized type of file that NTFS offers for efficient use when the file on the disk stream is mostly empty. Azure File Sync uses sparse files because a file is either fully tiered or partially recalled. In a fully tiered file, the file stream is stored in the cloud. In a partially recalled file, that part of the file is already on disk. This might occur when files are partially read by applications like multimedia players or zip utilities. If a file is fully recalled to disk, Azure File Sync converts it from a sparse file to a regular file. This attribute is only set on Windows Server 2016 and older.|
32
-
| M | Recall on data access | Indicates that the file's data is not fully present on local storage. Reading the file will cause at least some of the file content to be fetched from an Azure file share to which the server endpoint is connected. This attribute is only set on Windows Server 2019. |
32
+
| M | Recall on data access | Indicates that the file's data isn't fully present on local storage. Reading the file will cause at least some of the file content to be fetched from an Azure file share to which the server endpoint is connected. This attribute is only set on Windows Server 2019. |
33
33
| L | Reparse point | Indicates that the file has a reparse point. A reparse point is a special pointer for use by a file system filter. Azure File Sync uses reparse points to define to the Azure File Sync file system filter (StorageSync.sys) the cloud location where the file is stored. This supports seamless access. Users won't need to know that Azure File Sync is being used or how to get access to the file in your Azure file share. When a file is fully recalled, Azure File Sync removes the reparse point from the file. |
34
-
| O | Offline | Indicates that some or all of the file's content is not stored on disk. When a file is fully recalled, Azure File Sync removes this attribute. |
34
+
| O | Offline | Indicates that some or all of the file's content isn't stored on disk. When a file is fully recalled, Azure File Sync removes this attribute. |
35
35
36
36
37
37
@@ -43,7 +43,7 @@ There are several ways to check whether a file has been tiered to your Azure fil
43
43
> All of these attributes will be visible for partially recalled files as well.
44
44
45
45
-**Use `fsutil` to check for reparse points on a file.**
46
-
As described in the preceding option, a tiered file always has a reparse point set. A reparse point allows the Azure File Sync file system filter driver (StorageSync.sys) to retrieve content from Azure file shares that is not stored locally on the server.
46
+
As described in the preceding option, a tiered file always has a reparse point set. A reparse point allows the Azure File Sync file system filter driver (StorageSync.sys) to retrieve content from Azure file shares that isn't stored locally on the server.
47
47
48
48
To check whether a file has a reparse point, in an elevated Command Prompt or PowerShell window, run the `fsutil` utility:
49
49
@@ -54,13 +54,14 @@ There are several ways to check whether a file has been tiered to your Azure fil
54
54
If the file has a reparse point, you can expect to see **Reparse Tag Value: 0x8000001e**. This hexadecimal value is the reparse point value that is owned by Azure File Sync. The output also contains the reparse data that represents the path to your file on your Azure file share.
55
55
56
56
> [!WARNING]
57
-
> The `fsutil reparsepoint` utility command also has the ability to delete a reparse point. Do not execute this command unless the Azure File Sync engineering team asks you to. Running this command might result in data loss.
57
+
> The `fsutil reparsepoint` utility command also has the ability to delete a reparse point. Don't execute this command unless the Azure File Sync engineering team asks you to. Running this command might result in data loss.
58
58
59
59
## How to exclude files or folders from being tiered
60
60
61
-
If you want to exclude files or folders from being tiered and remain local on the Windows Server, you can configure the **GhostingExclusionList** registry setting under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Azure\StorageSync. You can exclude files by file name, file extension or path.
61
+
If you want to exclude files or folders from being tiered and remain local on the Windows Server, you can configure the **GhostingExclusionList** registry setting under `HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Azure\StorageSync`. You can exclude files by file name, file extension or path.
62
62
63
63
To exclude files or folders from cloud tiering, perform the following steps:
64
+
64
65
1. Open an elevated command prompt.
65
66
2. Run one of the following commands to configure exclusions:
66
67
@@ -70,47 +71,48 @@ To exclude files or folders from cloud tiering, perform the following steps:
70
71
To exclude a specific file name from tiering (for example, FileName.vhd), run the following command:
To exclude a combination of file names, file extensions and folders from tiering (for example, D:\ShareRoot\Folder1\SubFolder1,FileName.log,.txt), run the following command:
3. For the cloud tiering exclusions to take effect, you must restart the Storage Sync Agent service (FileSyncSvc) by running the following commands:
80
81
**net stop filesyncsvc**
81
82
**net start filesyncsvc**
82
83
83
84
### More information
84
-
- If the Azure File Sync agent is installed on a Failover Cluster, the **GhostingExclusionList** registry setting must be created under HKEY_LOCAL_MACHINE\Cluster\StorageSync\SOFTWARE\Microsoft\Azure\StorageSync.
85
+
86
+
- If the Azure File Sync agent is installed on a Failover Cluster, the **GhostingExclusionList** registry setting must be created under `HKEY_LOCAL_MACHINE\Cluster\StorageSync\SOFTWARE\Microsoft\Azure\StorageSync`.
- File name or file type exclusions apply to all server endpoints on the server.
90
-
- You cannot exclude file types from a particular folder only.
91
-
- Exclusions do not apply to files already tiered. Use the [Invoke-StorageSyncFileRecall](#how-to-recall-a-tiered-file-to-disk) cmdlet to recall files already tiered.
92
-
- Use Event ID 9001 in the Telemetry event log on the server to check the cloud tiering exclusions that are configured. The Telemetry event log is located in Event Viewer under Applications and Services\Microsoft\FileSync\Agent.
92
+
- You can't exclude file types from a particular folder only.
93
+
- Exclusions don't apply to files already tiered. Use the [Invoke-StorageSyncFileRecall](#how-to-recall-a-tiered-file-to-disk) cmdlet to recall files already tiered.
94
+
- Use Event ID 9001 in the Telemetry event log on the server to check the cloud tiering exclusions that are configured. The Telemetry event log is located in Event Viewer under `Applications and Services\Microsoft\FileSync\Agent`.
93
95
94
96
## How to exclude applications from cloud tiering last access time tracking
95
97
96
98
When an application accesses a file, the last access time for the file is updated in the cloud tiering database. Applications that scan the file system like anti-virus cause all files to have the same last access time, which impacts when files are tiered.
97
99
98
-
To exclude applications from last access time tracking, add the process exclusions to the **HeatTrackingProcessNamesExclusionList** registry setting under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Azure\StorageSync.
100
+
To exclude applications from last access time tracking, add the process exclusions to the **HeatTrackingProcessNamesExclusionList** registry setting under `HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Azure\StorageSync`.
If the Azure File Sync agent is installed on a Failover Cluster, the **HeatTrackingProcessNamesExclusionList** registry setting must be created under HKEY_LOCAL_MACHINE\Cluster\StorageSync\SOFTWARE\Microsoft\Azure\StorageSync.
104
+
If the Azure File Sync agent is installed on a Failover Cluster, the **HeatTrackingProcessNamesExclusionList** registry setting must be created under `HKEY_LOCAL_MACHINE\Cluster\StorageSync\SOFTWARE\Microsoft\Azure\StorageSync`.
> Data Deduplication and File Server Resource Manager (FSRM) processes are excluded by default. Changes to the process exclusion list are honored by the system every 5 minutes.
109
+
> Data Deduplication and File Server Resource Manager (FSRM) processes are excluded by default. Changes to the process exclusion list are honored by the system every five minutes.
108
110
109
111
## How to access the heat store
110
112
111
113
Cloud tiering uses the last access time and the access frequency of a file to determine which files should be tiered. The cloud tiering filter driver (storagesync.sys) tracks last access time and logs the information in the cloud tiering heat store. You can retrieve the heat store and save it into a CSV file by using a server-local PowerShell cmdlet.
112
114
113
-
There is a single heat store for all files on the same volume. The heat store can get very large. If you only need to retrieve the "coolest" number of items, use -Limit and a number and also consider filtering by a sub path vs. the volume root.
115
+
There is a single heat store for all files on the same volume. The heat store can get very large. If you only need to retrieve the "coolest" number of items, use -Limit and a number and also consider filtering by a sub path versus the volume root.
@@ -135,7 +137,7 @@ There is a single heat store for all files on the same volume. The heat store ca
135
137
> [!NOTE]
136
138
> When you select a directory to be tiered, only the files currently in the directory are tiered. Any files created after that time aren't automatically tiered.
137
139
138
-
When the cloud tiering feature is enabled, cloud tiering automatically tiers files based on last access and modify times to achieve the volume free space percentage specified on the cloud endpoint. Sometimes, though, you might want to manually force a file to tier. This might be useful if you save a large file that you don't intend to use again for a long time, and you want the free space on your volume now to use for other files and folders. You can force tiering by using the following PowerShell commands:
140
+
When the cloud tiering feature is enabled, cloud tiering automatically tiers files based on last access and modify times to achieve the volume free space percentage specified on the cloud endpoint. Sometimes you might want to manually force a file to tier. This might be useful if you save a large file that you don't intend to use again for a long time, and you want the free space on your volume now to use for other files and folders. You can force tiering by using the following PowerShell commands:
The easiest way to recall a file to disk is to open the file. The Azure File Sync file system filter (StorageSync.sys) seamlessly downloads the file from your Azure file share without any work on your part. For file types that can be partially read or streamed, such as multimedia or .zip files, simply opening a file doesn't ensure the entire file is downloaded.
149
+
The easiest way to recall a file to disk is to open the file. The Azure File Sync file system filter (StorageSync.sys) seamlessly downloads the file from your Azure file share. For file types that can be partially read or streamed, such as multimedia or .zip files, simply opening a file doesn't ensure the entire file is downloaded.
148
150
149
151
> [!NOTE]
150
-
> If a shortcut file is brought down to the server as a tiered file, there may be an issue when accessing the file over SMB. To mitigate this, there is task that runs every three days that will recall any shortcut files. However, if you would like shortcut files that are tiered to be recalled more frequently, create a scheduled task that runs this at the desired frequency:
152
+
> If a shortcut file is brought down to the server as a tiered file, there might be an issue when accessing the file over SMB. To mitigate this, there is a task that runs every three days that will recall any shortcut files. However, if you want shortcut files that are tiered to be recalled more frequently, create a scheduled task that runs this at the desired frequency:
-`-Order CloudTieringPolicy` will recall the most recently modified or accessed files first and is allowed by the current tiering policy.
165
-
* If volume free space policy is configured, files will be recalled until the volume free space policy setting is reached. For example if the volume free policy setting is 20%, recall will stop once the volume free space reaches 20%.
166
+
167
+
-`-Order CloudTieringPolicy` will recall the most recently modified or accessed files first, and is allowed by the current tiering policy.
168
+
* If volume free space policy is configured, files will be recalled until the volume free space policy setting is reached. For example, if the volume free policy setting is 20%, recall will stop once the volume free space reaches 20%.
166
169
* If volume free space and date policy is configured, files will be recalled until the volume free space or date policy setting is reached. For example, if the volume free policy setting is 20% and the date policy is 7 days, recall will stop once the volume free space reaches 20% or all files accessed or modified within 7 days are local.
167
170
-`-ThreadCount` determines how many files can be recalled in parallel (thread count limit is 32).
168
-
-`-PerFileRetryCount`determines how often a recall will be attempted of a file that is currently blocked.
169
-
-`-PerFileRetryDelaySeconds`determines the time in seconds between retry to recall attempts and should always be used in combination with the previous parameter.
171
+
-`-PerFileRetryCount`determines how often a recall will be attempted of a file that is currently blocked.
172
+
-`-PerFileRetryDelaySeconds`determines the time in seconds between retry to recall attempts and should always be used in combination with the previous parameter.
> - If the local volume hosting the server does not have enough free space to recall all the tiered data, the `Invoke-StorageSyncFileRecall` cmdlet fails.
182
+
> - If the local volume hosting the server doesn't have enough free space to recall all the tiered data, the `Invoke-StorageSyncFileRecall` cmdlet fails.
180
183
181
184
> [!NOTE]
182
-
> To recall files that have been tiered, the network bandwidth should be at least 1 Mbps. If network bandwidth is less than 1 Mbps, files may fail to recall with a timeout error.
185
+
> To recall files that have been tiered, the network bandwidth should be at least 1 Mbps. If network bandwidth is less than 1 Mbps, files might fail to recall with a timeout error.
0 commit comments