[Heap] Performance tweaks (apple#78)

* [Heap] Don't always inline large functions

* [docs] Use a stable link to the Atkinson article

(The original link was pointing to course materials at a random university.)

* Remove stray import

* Revert "[Heap] Don't always inline large functions"

This reverts commit 07ffd6591f2a790f6b84b668c06a188325d4eaf2.

* [Heap] Enable code coverage collection in Xcode scheme

* [Heap] Speed up invariant checking

* [Heap] Precalculate levels for each offset

`_Node` is a new struct that consists of a storage offset (the old index) along with its level in the tree. The level can be incrementally calculated, saving some time vs counting bits whenever it's needed.

* [Heap] Switch to using unsafe buffer pointers

Introduce `Heap._UnsafeHandle` (a thin wrapper around an unsafe buffer pointer) and rebase most heap algorithms on top of that instead of array operations.

This simplifies things by reducing (hopefully) unnecessary index validation, resulting in some measurable performance improvements.

* [Heap] Stop force-inlining bubbleUp; mark it releasenone

Not inlining such a large function speeds things up by leaving some headroom for the optimizer to make better inlining decisions elsewhere. (Force inlining this resulted in the compiler not inlining the closure passed to `_update` instead, which isn't a great tradeoff.

To speed things up, mark `bubbleUp` with `@_effects(releasenone)`. This may be questionable (because it calls `Element.<`), but it results in better codegen, making `insert` match the performance of `std::priority_queue`.

* [Heap] Rework removals

* [Heap] Remove dead code

* [Heap] Switch to using _ContiguousArray as storage

* [Heap] insert<S>(contentsOf:): add fast path for count == 0 case

* [Heap] Perf pass on trickleDown code paths

This improves popMin/popMax (and the sequence initializer) by reviewing trickleDown and optimizing things:

- Replace swapAt with a scheme where we keep a hold in the storage buffer
- Slightly shorten min/max dependency chain in primary sink loop

In exchange, we get even less readable code.

* [Heap] bubbleUp: remove @_effects attribute

This reintroduces retain/release operations for the empty array until swiftlang/swift#38898 lands.

Mitigate the performance costs of this by refactoring code to reduce the size of the inlined bubbleUpMin/Max invocations.

* [Heap] Finetune Heap.insert

Author: lorentey

.swiftpm/xcode/xcshareddata/xcschemes/PriorityQueueModule.xcscheme

@@ -26,7 +26,8 @@
       buildConfiguration = "Debug"
       selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
       selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
-      shouldUseLaunchSchemeArgsEnv = "YES">
+      shouldUseLaunchSchemeArgsEnv = "YES"
+      codeCoverageEnabled = "YES">
       <Testables>
          <TestableReference
             skipped = "NO">

Sources/PriorityQueueModule/Heap+Invariants.swift

@@ -14,78 +14,41 @@ import Foundation
 
 extension Heap {
   #if COLLECTIONS_INTERNAL_CHECKS
-  /// Iterates through all the levels in the heap, ensuring that items in min
-  /// levels are smaller than all their descendants and items in max levels
-  /// are larger than all their descendants.
-  ///
-  /// The min-max heap indices are structured like this:
-  ///
-  /// ```
-  /// min               0
-  /// max        1             2
-  /// min      3   4         5     6
-  /// max     7 8 9 10     11 12 13 14
-  /// min    15...
-  /// ...
-  /// ```
-  ///
-  /// The iteration happens in depth-first order, so the descendants of the
-  /// element at index 0 are checked to ensure they are >= that element. This is
-  /// repeated for each child of the element at index 0 (inverting the
-  /// comparison at each level).
-  ///
-  /// In the case of 7 elements total (spread across 3 levels), the checking
-  /// happens in the following order:
-  ///
-  /// ```
-  /// compare >= 0: 1, 3, 4, 2, 5, 6
-  /// compare <= 1: 3, 4
-  /// compare >= 3: (no children)
-  /// compare >= 4: (no children)
-  /// compare <= 2: 5, 6
-  /// compare >= 5: (no children)
-  /// compare >= 6: (no children)
-  /// ```
+  /// Visits each item in the heap in depth-first order, verifying that the
+  /// contents satisfy the min-max heap property.
   @inlinable
   @inline(never)
   internal func _checkInvariants() {
     guard count > 1 else { return }
-    var indicesToVisit: [Int] = [0]
-
-    while let elementIdx = indicesToVisit.popLast() {
-      let element = _storage[elementIdx]
-
-      let isMinLevel = _minMaxHeapIsMinLevel(elementIdx)
-
-      var descendantIndicesToVisit = [Int]()
+    _checkInvariants(node: .root, min: nil, max: nil)
+  }
 
-      // Add the children of this element to the outer loop (as we want to check
-      // that they are >= or <= their descendants as well)
-      if let rightIdx = _rightChildIndex(of: elementIdx) {
-        descendantIndicesToVisit.append(rightIdx)
-        indicesToVisit.append(rightIdx)
+  @inlinable
+  internal func _checkInvariants(node: _Node, min: Element?, max: Element?) {
+    let value = _storage[node.offset]
+    if let min = min {
+      precondition(value >= min,
+                   "Element \(value) at \(node) is less than min \(min)")
+    }
+    if let max = max {
+      precondition(value <= max,
+                   "Element \(value) at \(node) is greater than max \(max)")
+    }
+    let left = node.leftChild()
+    let right = node.rightChild()
+    if node.isMinLevel {
+      if left.offset < count {
+        _checkInvariants(node: left, min: value, max: max)
       }
-      if let leftIdx = _leftChildIndex(of: elementIdx) {
-        descendantIndicesToVisit.append(leftIdx)
-        indicesToVisit.append(leftIdx)
+      if right.offset < count {
+        _checkInvariants(node: right, min: value, max: max)
       }
-
-      // Compare the current element against its descendants
-      while let idx = descendantIndicesToVisit.popLast() {
-        if isMinLevel {
-          precondition(element <= _storage[idx],
-            "Element '\(_storage[idx])' at index \(idx) should be >= '\(element)'")
-        } else {
-          precondition(element >= _storage[idx],
-            "Element '\(_storage[idx])' at index \(idx) should be <= '\(element)'")
-        }
-
-        if let rightIdx = _rightChildIndex(of: idx) {
-          descendantIndicesToVisit.append(rightIdx)
-        }
-        if let leftIdx = _leftChildIndex(of: idx) {
-          descendantIndicesToVisit.append(leftIdx)
-        }
+    } else {
+      if left.offset < count {
+        _checkInvariants(node: left, min: min, max: value)
+      }
+      if right.offset < count {
+        _checkInvariants(node: right, min: min, max: value)
       }
     }
   }