Merge pull request #284250 from b-ahibbard/directory-size

Directory size

Author: denrea

articles/azure-netapp-files/TOC.yml

@@ -67,6 +67,12 @@
       href: azure-netapp-files-network-topologies.md
     - name: Resource limits for Azure NetApp Files
       href: azure-netapp-files-resource-limits.md
+    - name: High file counts 
+      items: 
+        - name: Understand maxfiles 
+          href: maxfiles-concept.md
+        - name: Understand maximum directory sizes 
+          href: directory-sizes-concept.md
     - name: Ways to monitor Azure NetApp Files
       href: monitor-azure-netapp-files.md
     - name: Metrics for Azure NetApp Files

articles/azure-netapp-files/azure-netapp-files-resize-capacity-pools-or-volumes.md

@@ -18,7 +18,7 @@ For information about monitoring a volume’s capacity, see [Monitor the capacit
 ## Considerations
 
 * Resize operations on Azure NetApp Files volumes don't result in data loss.
-* Volume quotas are indexed against `maxfiles` limits. Once a volume has surpassed a `maxfiles` limit, you cannot reduce the volume size below the quota that corresponds to that `maxfiles` limit. For more information and specific limits, see [`maxfiles` limits](azure-netapp-files-resource-limits.md#maxfiles-limits-).
+* Volume quotas are indexed against `maxfiles` limits. Once a volume has surpassed a `maxfiles` limit, you cannot reduce the volume size below the quota that corresponds to that `maxfiles` limit. For more information and specific limits, see [`maxfiles` limits](maxfiles-concept.md).
 * Capacity pools with Basic network features have a minimum size of 4 TiB. For capacity pools with Standard network features, the minimum size is 1 TiB. For more information, see [Resource limits](azure-netapp-files-resource-limits.md)
 * Volume resize operations are nearly instantaneous but not always immediate. There can be a short delay for the volume's updated size to appear in the portal. Verify the size from a host perspective before re-attempting the resize operation.
 

articles/azure-netapp-files/azure-netapp-files-resource-limits.md

@@ -5,7 +5,7 @@ services: azure-netapp-files
 author: b-hchen
 ms.service: azure-netapp-files
 ms.topic: conceptual
-ms.date: 07/23/2024
+ms.date: 08/09/2024
 ms.author: anfdocs
 ---
 # Resource limits for Azure NetApp Files
@@ -35,8 +35,8 @@ The following table describes resource limits for Azure NetApp Files:
 | Maximum size of a single large volume on dedicated capacity (preview) | 2,048 TiB | No |
 |  Maximum size of a single file     |    16 TiB    |    No    |    
 |  Maximum size of directory metadata in a single directory      |    320 MB    |    No    |    
-|  Maximum number of files in a single directory  | *Approximately* 4 million. <br> See [Determine if a directory is approaching the limit size](#directory-limit).  |    No    |   
-|  Maximum number of files `maxfiles` per volume     |  See [`maxfiles`](#maxfiles)  |    Yes    |    
+|  Maximum number of files in a single directory  | *Approximately* 4 million. <br> See [Determine if a directory is approaching the limit size](directory-sizes-concept.md#directory-limit).  |    No    |   
+|  Maximum number of files `maxfiles` per volume     |  See [`maxfiles`](maxfiles-concept.md)  |    Yes    |    
 |  Maximum number of export policy rules per volume     |    5  |    No    | 
 |  Maximum number of quota rules per volume     |   100  |    No    | 
 |  Minimum assigned throughput for a manual QoS volume     |    1 MiB/s   |    No    |    
@@ -56,67 +56,6 @@ For more information, see [Capacity management FAQs](faq-capacity-management.md)
 
 For limits and constraints related to Azure NetApp Files network features, see [Guidelines for Azure NetApp Files network planning](azure-netapp-files-network-topologies.md#considerations).
 
-## Determine if a directory is approaching the limit size <a name="directory-limit"></a>  
-
-You can use the `stat` command from a client to see whether a directory is approaching the maximum size limit for directory metadata (320 MB). If you reach the maximum size limit for a single directory for Azure NetApp Files, the error `No space left on device` occurs.   
-
-For a 320-MB directory, the number of blocks is 655,360, with each block size being 512 bytes.  (That is, 320x1024x1024/512.)  This number translates to approximately 4 million files maximum for a 320-MB directory. However, the actual number of maximum files might be lower, depending on factors such as the number of files with non-ASCII characters in the directory. As such, you should use the `stat` command as follows to determine whether your directory is approaching its limit.  
-
-Examples:
-
-```console
-[makam@cycrh6rtp07 ~]$ stat bin
-File: 'bin'
-Size: 4096            Blocks: 8          IO Block: 65536  directory
-
-[makam@cycrh6rtp07 ~]$ stat tmp
-File: 'tmp'
-Size: 12288           Blocks: 24         IO Block: 65536  directory
- 
-[makam@cycrh6rtp07 ~]$ stat tmp1
-File: 'tmp1'
-Size: 4096            Blocks: 8          IO Block: 65536  directory
-```
-
-## `Maxfiles` limits <a name="maxfiles"></a> 
-
-Azure NetApp Files volumes have a value called `maxfiles` that refers to the maximum number of files and folders (also known as inodes) a volume can contain. When the `maxfiles` limit is reached, clients receive "out of space" messages when attempting to create new files or folders. If you experience this issue, contact Microsoft technical support.
-
-The `maxfiles` limit for an Azure NetApp Files volume is based on the size (quota) of the volume, where the service dynamically adjusts the `maxfiles` limit for a volume based on its provisioned size and uses the following guidelines.
-
-- For regular volumes less than or equal to 683 GiB, the default `maxfiles` limit is 21,251,126.
-- For regular volumes greater than 683 GiB, the default `maxfiles` limit is approximately one file (or inode) per 32 KiB of allocated volume capacity up to a maximum of 2,147,483,632.
-- For [large volumes](large-volumes-requirements-considerations.md), the default `maxfiles` limit is approximately one file (or inode) per 32 KiB of allocated volume capacity up to a default maximum of 15,938,355,048.
-- Each inode uses roughly 288 bytes of capacity in the volume. Having many inodes in a volume can consume a non-trivial amount of physical space overhead on top of the capacity of the actual data.
-    - If a file is less than 64 bytes in size, it's stored in the inode itself and doesn't use additional capacity. This capacity is only used when files are actually allocated to the volume.
-    - Files larger than 64 bytes do consume additional capacity on the volume. For instance, if there are one million files greater than 64 bytes in an Azure NetApp Files volume, then approximately 274 MiB of capacity would belong to the inodes.
-
-
-The following table shows examples of the relationship `maxfiles` values based on volume sizes for regular volumes. 
-
-| Volume size | Estimated maxfiles limit |
-| - | - |
-| 0 – 683 GiB | 21,251,126 |
-| 1 TiB (1,073,741,824 KiB) | 31,876,709 |
-| 10 TiB (10,737,418,240 KiB) | 318,767,099 |
-| 50 TiB (53,687,091,200 KiB)	| 1,593,835,519 |
-| 100 TiB (107,374,182,400 KiB) | 2,147,483,632 |
-
-The following table shows examples of the relationship `maxfiles` values based on volume sizes for large volumes. 
-
-| Volume size | Estimated maxfiles limit |
-| - | - |
-| 50 TiB (53,687,091,200 KiB) |	1,593,835,512 |
-| 100 TiB (107,374,182,400 KiB)	| 3,187,671,024 |
-| 200 TiB (214,748,364,800 KiB)	| 6,375,342,024  |
-| 500 TiB (536,870,912,000 KiB)	| 15,938,355,048 |
-
-To see the `maxfiles` allocation for a specific volume size, check the **Maximum number of files** field in the volume’s overview pane.
-
-:::image type="content" source="./media/azure-netapp-files-resource-limits/maximum-number-files.png" alt-text="Screenshot of volume overview menu." lightbox="./media/azure-netapp-files-resource-limits/maximum-number-files.png":::
-
-You can't set `maxfiles` limits for data protection volumes via a quota request. Azure NetApp Files automatically increases the `maxfiles` limit of a data protection volume to accommodate the number of files replicated to the volume. When a failover happens on a data protection volume, the `maxfiles` limit remains the last value before the failover. In this situation, you can submit a `maxfiles` [quota request](#request-limit-increase) for the volume.
-
 ## Request limit increase
 
 You can create an Azure support request to increase the adjustable limits from the [Resource Limits](#resource-limits) table. 
@@ -155,6 +94,8 @@ You can create an Azure support request to increase the adjustable limits from t
 
 ## Next steps  
 
+- [Understand `maxfiles` limits](maxfiles-concept.md)
+- [Understand maximum directory sizes](directory-sizes-concept.md)
 - [Understand the storage hierarchy of Azure NetApp Files](azure-netapp-files-understand-storage-hierarchy.md)
 - [Requirements and considerations for large volumes](large-volumes-requirements-considerations.md)
 - [Cost model for Azure NetApp Files](azure-netapp-files-cost-model.md)

articles/azure-netapp-files/directory-sizes-concept.md

@@ -0,0 +1,124 @@
+---
+title: Understand directory sizes in Azure NetApp Files 
+description: Learn how metadata impacts directory sizes in Azure NetApp Files. 
+services: azure-netapp-files
+author: b-ahibbard
+ms.service: azure-netapp-files
+ms.topic: conceptual
+ms.date: 07/23/2024
+ms.author: anfdocs
+---
+# Understand directory sizes in Azure NetApp Files 
+
+When a file is created in a directory, an entry is added to a hidden index file within the Azure NetApp Files volume. This index file helps keep track of the existing inodes in a directory and helps expedite lookup requests for directories with a high number of files. As entries are added to this file, the file size increases (but never decrease) at a rate of approximately 512 bytes per entry depending on the length of the filename. Longer file names add more size to the file. Symbolic links also add entries to this file. This concept is known as the directory size, which is a common element in all Linux-based filesystems. Directory size isn't the maximum total number of files in a single Azure NetApp Files volume. That is determined by the [`maxfiles` value](maxfiles-concept.md). 
+
+By default, when a new directory is created, it consumes 4 KiB (4,096 bytes) or eight 512-byte blocks. You can view the size of a newly created directory from a Linux client using the stat command. 
+
+```
+# mkdir dirsize 
+# stat dirsize 
+File: ‘dirsize’ 
+Size: 4096            Blocks: 8          IO Block: 32768  directory 
+``` 
+
+Directory sizes are specific to a single directory and don't combine in sizes. For example, if there are 10 directories in a volume, each can approach the 320-MiB directory size limit in a single volume. 
+
+## Determine if a directory is approaching the limit size <a name="directory-limit"></a>  
+
+For a 320-MiB directory, the number of blocks is 655360, with each block size being 512 bytes. (That is, 320x1024x1024/512.) This number translates to approximately 4-5 million files maximum for a 320-MiB directory. However, the actual number of maximum files might be lower, depending on factors such as the number of files with non-ASCII characters in the directory. 
+You can use the `stat` command from a client to see whether a directory is approaching the maximum size limit for directory metadata (320 MB). If you reach the maximum size limit for a single directory for Azure NetApp Files, the error `No space left on device` occurs.   
+
+For a 320-MB directory, the number of blocks is 655,360, with each block size being 512 bytes.  (That is, 320x1024x1024/512.)  This number translates to approximately 4 million files maximum for a 320-MB directory. However, the actual number of maximum files might be lower, depending on factors such as the number of files with non-ASCII characters in the directory. For information on how to monitor the maxdirsize, see [Monitoring `maxdirsize`]().
+
+## Directory size considerations 
+
+When dealing with a high-file count environment, consider the following recommendations: 
+
+- Azure NetApp Files volumes support up to 320 MiB for directory sizes. This value can't be increased. 
+- Once a volume’s directory size has been exceeded, clients display an out-of-space error even if there's available free space in the volume.  
+- For regular volumes, a 320 MiB directory size equates to roughly 4-5 million files in a single directory. This value is dependent on the file name lengths. 
+- [Large volumes](large-volumes-requirements-considerations.md) have a different architecture than regular volumes.
+- High file counts in a single directory can present performance problems when searching. When possible, limit the total size of a single directory to 2 MiB (roughly 27,000 files) when frequent searches are needed.  
+    - If more files are needed in a single directory, adjust search performance expectations accordingly. While Azure NetApp Files indexes the directory file listings for performance, searches can  take some time with high file counts. 
+- When designing your file system, avoid flat directory layouts. For information about different approaches to directory layouts, see [About directory layouts](#about-directory-layouts).
+- To resolve issues where the directory size has been exceeded and new files can't be created, delete or move files out of the relevant directory.
+
+## About directory layouts
+
+The `maxdirsize` value can create concerns when you're using flat directory structures, where a single folder contains millions of files at a single level. Folder structures where files, folders, and subfolders are interspersed have a low impact on `maxdirsize`. There are several directory structure methodologies. 
+
+A **flat directory structure** is a single directory with many files below the same directory. 
+
+:::image type="content" source="./media/directory-sizes-concept/flat-structure.png" alt-text="Diagram of a flat directory structure.":::
+
+A **wide directory structure** contains many top-level directories with files spread across all directories.
+
+:::image type="content" source="./media/directory-sizes-concept/wide-structure.png" alt-text="Diagram of a wide directory structure.":::
+
+A **deep directory structure** contains fewer top-level directories with many subdirectories. Although this structure provides fewer files per folder, file path lengths can become an issue if directory layouts are too deep and file paths become too long. For details on file path lengths, see [Understand file path lengths in Azure NetApp Files](understand-path-lengths.md).
+
+:::image type="content" source="./media/directory-sizes-concept/deep-structure.png" alt-text="Diagram of a deep directory structure.":::
+
+### Impact of flat directory structures in Azure NetApp Files
+
+Flat directory structures (many files in a single or few directories) have a negative effect on a wide array of file systems, Azure NetApp File volumes, or others. Potential issues include:
+
+- Memory pressure
+- CPU utilization
+- Network performance/latency (especially during mass queries of files, `GETATTR` operations, `READDIR` operations)
+
+Due to the design of Azure NetApp Files large volumes, the impact of `maxdirsize` is unique. Azure NetApp Files large volume `maxdirsize` is impacted uniquely due to its design. Unlike a regular volume, a large volume uses remote hard links inside Azure NetApp Files to help redirect traffic across different storage devices to provide more scale and performance. When using flat directories, there's a higher ratio of internal remote hard links to local files. These remote hard links count against the total `maxdirsize` value, so a large volume might approach its `maxdirsize` limit faster than a regular volume.
+
+For example, if a single directory has millions of files and generates roughly 85% remote hard links for the file system, you can expect `maxdirsize` to be exhausted at nearly twice the amount as a regular volume would.
+
+For best results with directory sizes in Azure NetApp Files:
+
+- **Avoid flat directory structures in Azure NetApp Files**. **Wide or deep directory structures work best**, provided the [path length](understand-path-lengths.md) of the file or folder doesn't exceed NAS protocol standards. 
+- If flat directory structures are unavoidable, monitor the `maxdirsize` for the directories.
+
+## Monitor `maxdirsize`
+
+For a single directory, use the `stat` command to find the directory size. 
+
+```
+# stat /mnt/dir_11/c5 
+```
+ 
+Although the `stat` command can be used to check the directory size of a specific directory, it might not be as efficient to run it individually against a single directory. To see a list of the largest directory sizes sorted from largest to smallest, the following command provides that while omitting snapshot directories from the query. 
+
+```
+# find /mnt -name .snapshot -prune -o -type d -ls -links 2 -prune | sort -rn -k 7 | head | awk '{print $2 " " $11}' | sort -rn 
+```
+
+>[!NOTE]
+>The directory size reported by the stat command is in bytes. The size reported in the find command is in KiB.
+ 
+**Example**
+```
+# stat /mnt/dir_11/c5 
+
+  File: ‘/mnt/dir_11/c5’ 
+
+  Size: 322396160       Blocks: 632168     IO Block: 32768  directory 
+ 
+# find /mnt -name .snapshot -prune -o -type d -ls -links 2 -prune | sort -rn -k 7 | head | awk '{print $2 " " $11}' | sort -rn 
+316084 /mnt/dir_11/c5 
+
+3792 /mnt/dir_19 
+
+3792 /mnt/dir_16 
+```
+
+In the previous, the directory size of `/mnt/dir_11/c5` is 316,084 KiB (308.6 MiB), which approaches the 320-MiB limit. That equates to around 4.1 million files.
+
+```
+# ls /mnt/dir_11/c5 | wc -l
+4171624
+```
+
+In this case, consider corrective actions such as moving or deleting files.
+
+## More information 
+
+* [Azure NetApp Files resources limits](azure-netapp-files-resource-limits.md)
+* [Understand file path lengths in Azure NetApp Files](understand-path-lengths.md)

articles/azure-netapp-files/faq-capacity-management.md

@@ -18,7 +18,8 @@ Azure NetApp Files provides capacity pool and volume usage metrics. You can also
 ## How do I determine if a directory is approaching the limit size?
 
 You can use the `stat` command from a client to see whether a directory is approaching the [maximum size limit](azure-netapp-files-resource-limits.md#resource-limits) for directory metadata (320 MB).
-See [Resource limits for Azure NetApp Files](azure-netapp-files-resource-limits.md#directory-limit) for the limit and calculation. 
+
+See [Understand directory sizes in Azure NetApp Files](directory-sizes-concept.md) for the limit and calculation. 
 
 ## Does snapshot space count towards the usable / provisioned capacity of a volume?
 
@@ -29,17 +30,17 @@ Yes, the [consumed snapshot capacity](azure-netapp-files-cost-model.md#capacity-
 
 ## Does Azure NetApp Files support auto-grow for volumes or capacity pools?
 
-No, Azure NetApp Files volumes and capacity pool do not auto-grow upon filling up. See [Cost model for Azure NetApp Files](azure-netapp-files-cost-model.md).   
+No, Azure NetApp Files volumes and capacity pool don't auto-grow upon filling up. See [Cost model for Azure NetApp Files](azure-netapp-files-cost-model.md).   
 
 You can use the community supported [Logic Apps ANFCapacityManager tool](https://github.com/ANFTechTeam/ANFCapacityManager) to manage capacity-based alert rules. The tool can automatically increase volume sizes to prevent your volumes from running out of space.
 
 ## Does the destination volume of a replication count towards hard volume quota?  
 
-No, the destination volume of a replication does not count towards hard volume quota.
+No, the destination volume of a replication doesn't count towards hard volume quota.
 
 ## Can I manage Azure NetApp Files through Azure Storage Explorer?
 
-No. Azure NetApp Files is not supported by Azure Storage Explorer.
+No. Azure NetApp Files isn't supported by Azure Storage Explorer.
 
 ## Why is volume space not freed up immediately after deleting large amount of data in a volume?