Fixed ArrayPoolBufferWriter<T> repeated new[] allocations (#3524)

## PR Type
What kind of change does this PR introduce?
<!-- Please uncomment one or more that apply to this PR. -->

- Optimization
<!-- - Bugfix -->
<!-- - Feature -->
<!-- - Code style update (formatting) -->
<!-- - Refactoring (no functional changes, no api changes) -->
<!-- - Build or CI related changes -->
<!-- - Documentation content changes -->
<!-- - Sample app changes -->
<!-- - Other... Please describe: -->


## What is the current behavior?
<!-- Please describe the current behavior that you are modifying, or link to a relevant issue. -->
The `ArrayPoolBufferWriter<T>` uses the given `ArrayPool<T>` instance to resize its internal buffer when needed, which only works as expected when the array itself is small. The issue is that the `ArrayPool<T>.Shared` instance has an internal threshold set to `1024 * 1024`, over which it just allocates new arrays every time to avoid keeping very large arrays alive for a long time. That is perfectly fine, except for one little detail: once you get past that threshold, `ArrayPool<T>.Shared` **stops rounding up the requested size**. It instead returns an array with `new[]` of **exactly the requested size**, which absolutely kills the performance when used in a writer type like `ArrayPoolBufferWriter<T>`: this means that as soon as we get past tha threshold, we'll basically end up resizing the whole array for every single new write operation, no matter how large it is. That's super bad for performance and memory usage 🥺

## What is the new behavior?
<!-- Describe how was this issue resolved or changed? -->
The solution for this is pretty simple, this PR includes a simple check for the requested size, and if that's over `1024 * 1024` it just rounds that up to the closest power of 2, so that the array size will effectively just keep being multiplied by 2 every time. This has a **huge** performance impact when eg. trying to use the `ArrayPoolBufferWriter<T>` class to write a 10MB buffer, 8KB at a time:

|                    Method |         Mean |      Error |     StdDev | Ratio |       Gen 0 |       Gen 1 |       Gen 2 |  Allocated |
|-------------------------- |-------------:|-----------:|-----------:|------:|------------:|------------:|------------:|-----------:|
| Before | 1,380.236 ms | 19.9806 ms | 17.7122 ms | 1.000 | 355000.0000 | 355000.0000 | 355000.0000 | 5754.44 MB |
| After |     **8.820 ms** |  0.1757 ms |  0.4838 ms | **0.006** |    640.6250 |    640.6250 |    640.6250 |      **30 MB** |

In this simple benchmark alone, the updated version is **156x** faster and uses **190x less memory** 😄
Of course, results will vary a lot on the specific workload, but you can imagine the impact being even more dramatic when working with larger buffers, or with less items being written at any given time. With this change in general, users will not have to worry about the size of the data being written, and the class will automatically use the right approach in all cases.

## PR Checklist

Please check if your PR fulfills the following requirements:

- [X] Tested code with current [supported SDKs](../readme.md#supported)
- [ ] ~~Pull Request has been submitted to the documentation repository [instructions](..\contributing.md#docs). Link: <!-- docs PR link -->~~
- [ ] ~~Sample in sample app has been added / updated (for bug fixes / features)~~
    - [ ] ~~Icon has been created (if new sample) following the [Thumbnail Style Guide and templates](https://github.com/windows-toolkit/WindowsCommunityToolkit-design-assets)~~
- [X] Tests for the changes have been added (for bug fixes / features) (if applicable)
- [X] Header has been added to all new source files (run *build/UpdateHeaders.bat*)
- [X] Contains **NO** breaking changes

Author: msftbot[bot]

Microsoft.Toolkit.HighPerformance/Buffers/ArrayPoolBufferWriter{T}.cs

@@ -10,6 +10,7 @@
 using System.Runtime.InteropServices;
 using Microsoft.Toolkit.HighPerformance.Buffers.Views;
 using Microsoft.Toolkit.HighPerformance.Extensions;
+using Microsoft.Toolkit.HighPerformance.Helpers.Internals;
 
 namespace Microsoft.Toolkit.HighPerformance.Buffers
 {
@@ -233,15 +234,15 @@ public void Advance(int count)
         /// <inheritdoc/>
         public Memory<T> GetMemory(int sizeHint = 0)
         {
-            CheckAndResizeBuffer(sizeHint);
+            CheckBufferAndEnsureCapacity(sizeHint);
 
             return this.array.AsMemory(this.index);
         }
 
         /// <inheritdoc/>
         public Span<T> GetSpan(int sizeHint = 0)
         {
-            CheckAndResizeBuffer(sizeHint);
+            CheckBufferAndEnsureCapacity(sizeHint);
 
             return this.array.AsSpan(this.index);
         }
@@ -251,9 +252,11 @@ public Span<T> GetSpan(int sizeHint = 0)
         /// </summary>
         /// <param name="sizeHint">The minimum number of items to ensure space for in <see cref="array"/>.</param>
         [MethodImpl(MethodImplOptions.AggressiveInlining)]
-        private void CheckAndResizeBuffer(int sizeHint)
+        private void CheckBufferAndEnsureCapacity(int sizeHint)
         {
-            if (this.array is null)
+            T[]? array = this.array;
+
+            if (array is null)
             {
                 ThrowObjectDisposedException();
             }
@@ -268,12 +271,32 @@ private void CheckAndResizeBuffer(int sizeHint)
                 sizeHint = 1;
             }
 
-            if (sizeHint > FreeCapacity)
+            if (sizeHint > array!.Length - this.index)
             {
-                int minimumSize = this.index + sizeHint;
+                ResizeBuffer(sizeHint);
+            }
+        }
 
-                this.pool.Resize(ref this.array, minimumSize);
+        /// <summary>
+        /// Resizes <see cref="array"/> to ensure it can fit the specified number of new items.
+        /// </summary>
+        /// <param name="sizeHint">The minimum number of items to ensure space for in <see cref="array"/>.</param>
+        [MethodImpl(MethodImplOptions.NoInlining)]
+        private void ResizeBuffer(int sizeHint)
+        {
+            int minimumSize = this.index + sizeHint;
+
+            // The ArrayPool<T> class has a maximum threshold of 1024 * 1024 for the maximum length of
+            // pooled arrays, and once this is exceeded it will just allocate a new array every time
+            // of exactly the requested size. In that case, we manually round up the requested size to
+            // the nearest power of two, to ensure that repeated consecutive writes when the array in
+            // use is bigger than that threshold don't end up causing a resize every single time.
+            if (minimumSize > 1024 * 1024)
+            {
+                minimumSize = BitOperations.RoundUpPowerOfTwo(minimumSize);
             }
+
+            this.pool.Resize(ref this.array, minimumSize);
         }
 
         /// <inheritdoc/>

Microsoft.Toolkit.HighPerformance/Buffers/MemoryOwner{T}.cs

@@ -244,6 +244,11 @@ public MemoryOwner<T> Slice(int start, int length)
                 ThrowInvalidLengthException();
             }
 
+            // We're transferring the ownership of the underlying array, so the current
+            // instance no longer needs to be disposed. Because of this, we can manually
+            // suppress the finalizer to reduce the overhead on the garbage collector.
+            GC.SuppressFinalize(this);
+
             return new MemoryOwner<T>(start, length, this.pool, array!);
         }
 

Microsoft.Toolkit.HighPerformance/Buffers/StringPool.cs

@@ -5,15 +5,13 @@
 using System;
 using System.Diagnostics.CodeAnalysis;
 using System.Diagnostics.Contracts;
-#if NETCOREAPP3_1
-using System.Numerics;
-#endif
 using System.Runtime.CompilerServices;
 using System.Text;
 using Microsoft.Toolkit.HighPerformance.Extensions;
 #if !NETSTANDARD1_4
 using Microsoft.Toolkit.HighPerformance.Helpers;
 #endif
+using BitOperations = Microsoft.Toolkit.HighPerformance.Helpers.Internals.BitOperations;
 
 #nullable enable
 
@@ -79,8 +77,8 @@ static void FindFactors(int size, int factor, out int x, out int y)
                     a = Math.Sqrt((double)size / factor),
                     b = factor * a;
 
-                x = RoundUpPowerOfTwo((int)a);
-                y = RoundUpPowerOfTwo((int)b);
+                x = BitOperations.RoundUpPowerOfTwo((int)a);
+                y = BitOperations.RoundUpPowerOfTwo((int)b);
             }
 
             // We want to find two powers of 2 factors that produce a number
@@ -130,30 +128,6 @@ static void FindFactors(int size, int factor, out int x, out int y)
             Size = p2;
         }
 
-        /// <summary>
-        /// Rounds up an <see cref="int"/> value to a power of 2.
-        /// </summary>
-        /// <param name="x">The input value to round up.</param>
-        /// <returns>The smallest power of two greater than or equal to <paramref name="x"/>.</returns>
-        [Pure]
-        [MethodImpl(MethodImplOptions.AggressiveInlining)]
-        private static int RoundUpPowerOfTwo(int x)
-        {
-#if NETCOREAPP3_1
-            return 1 << (32 - BitOperations.LeadingZeroCount((uint)(x - 1)));
-#else
-            x--;
-            x |= x >> 1;
-            x |= x >> 2;
-            x |= x >> 4;
-            x |= x >> 8;
-            x |= x >> 16;
-            x++;
-
-            return x;
-#endif
-        }
-
         /// <summary>
         /// Gets the shared <see cref="StringPool"/> instance.
         /// </summary>

Microsoft.Toolkit.HighPerformance/Helpers/Internals/BitOperations.cs

@@ -0,0 +1,43 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+// See the LICENSE file in the project root for more information.
+
+using System.Diagnostics.Contracts;
+using System.Runtime.CompilerServices;
+#if NETCOREAPP3_1
+using static System.Numerics.BitOperations;
+#endif
+
+namespace Microsoft.Toolkit.HighPerformance.Helpers.Internals
+{
+    /// <summary>
+    /// Utility methods for intrinsic bit-twiddling operations. The methods use hardware intrinsics
+    /// when available on the underlying platform, otherwise they use optimized software fallbacks.
+    /// </summary>
+    internal static class BitOperations
+    {
+        /// <summary>
+        /// Rounds up an <see cref="int"/> value to a power of 2.
+        /// </summary>
+        /// <param name="x">The input value to round up.</param>
+        /// <returns>The smallest power of two greater than or equal to <paramref name="x"/>.</returns>
+        [Pure]
+        [MethodImpl(MethodImplOptions.AggressiveInlining)]
+        public static int RoundUpPowerOfTwo(int x)
+        {
+#if NETCOREAPP3_1
+            return 1 << (32 - LeadingZeroCount((uint)(x - 1)));
+#else
+            x--;
+            x |= x >> 1;
+            x |= x >> 2;
+            x |= x >> 4;
+            x |= x >> 8;
+            x |= x >> 16;
+            x++;
+
+            return x;
+#endif
+        }
+    }
+}

UnitTests/UnitTests.HighPerformance.Shared/Buffers/Test_ArrayPoolBufferWriter{T}.cs

@@ -6,6 +6,7 @@
 using System.Diagnostics.CodeAnalysis;
 using System.IO;
 using System.Linq;
+using System.Reflection;
 using Microsoft.Toolkit.HighPerformance.Buffers;
 using Microsoft.Toolkit.HighPerformance.Extensions;
 using Microsoft.VisualStudio.TestTools.UnitTesting;
@@ -17,6 +18,44 @@ namespace UnitTests.HighPerformance.Buffers
     [SuppressMessage("StyleCop.CSharp.DocumentationRules", "SA1649", Justification = "Test class for generic type")]
     public class Test_ArrayPoolBufferWriterOfT
     {
+        [TestCategory("ArrayPoolBufferWriterOfT")]
+        [TestMethod]
+        [DataRow(0, 256)] // 256 is the default initial size for ArrayPoolBufferWriter<T>
+        [DataRow(4, 256)]
+        [DataRow(7, 256)]
+        [DataRow(27, 256)]
+        [DataRow(188, 256)]
+        [DataRow(257, 512)]
+        [DataRow(358, 512)]
+        [DataRow(799, 1024)]
+        [DataRow(1024, 1024)]
+        [DataRow(1025, 2048)]
+        [DataRow((1024 * 1024) - 1, 1024 * 1024)]
+        [DataRow(1024 * 1024, 1024 * 1024)]
+        [DataRow((1024 * 1024) + 1, 2 * 1024 * 1024)]
+        [DataRow(2 * 1024 * 1024, 2 * 1024 * 1024)]
+        [DataRow((2 * 1024 * 1024) + 1, 4 * 1024 * 1024)]
+        [DataRow(3 * 1024 * 1024, 4 * 1024 * 1024)]
+        public void Test_ArrayPoolBufferWriterOfT_BufferSize(int request, int expected)
+        {
+            using var writer = new ArrayPoolBufferWriter<byte>();
+
+            // Request a Span<T> of a specified size and discard it. We're just invoking this
+            // method to force the ArrayPoolBufferWriter<T> instance to internally resize the
+            // buffer to ensure it can contain at least this number of items. After this, we
+            // can use reflection to get the internal array and ensure the size equals the
+            // expected one, which matches the "round up to power of 2" logic we need. This
+            // is documented within the resize method in ArrayPoolBufferWriter<T>, and it's
+            // done to prevent repeated allocations of arrays in some scenarios.
+            _ = writer.GetSpan(request);
+
+            var arrayFieldInfo = typeof(ArrayPoolBufferWriter<byte>).GetField("array", BindingFlags.Instance | BindingFlags.NonPublic);
+
+            byte[] array = (byte[])arrayFieldInfo!.GetValue(writer);
+
+            Assert.AreEqual(array!.Length, expected);
+        }
+
         [TestCategory("ArrayPoolBufferWriterOfT")]
         [TestMethod]
         public void Test_ArrayPoolBufferWriterOfT_AllocateAndGetMemoryAndSpan()