|
| 1 | +# Range-based confirmations |
| 2 | + |
| 3 | +* Proposal: [SWT-0005](0005-ranged-confirmations.md) |
| 4 | +* Authors: [Jonathan Grynspan](https://github.com/grynspan) |
| 5 | +* Status: **Accepted** |
| 6 | +* Bug: rdar://138499457 |
| 7 | +* Implementation: [swiftlang/swift-testing#598](https://github.com/swiftlang/swift-testing/pull/598), [swiftlang/swift-testing#689](https://github.com/swiftlang/swift-testing/pull689) |
| 8 | +* Review: ([pitch](https://forums.swift.org/t/pitch-range-based-confirmations/74589)), |
| 9 | + ([acceptance](https://forums.swift.org/t/pitch-range-based-confirmations/74589/7)) |
| 10 | + |
| 11 | +## Introduction |
| 12 | + |
| 13 | +Swift Testing includes [an interface](https://swiftpackageindex.com/swiftlang/swift-testing/main/documentation/testing/confirmation(_:expectedcount:isolation:sourcelocation:_:)) |
| 14 | +for checking that some asynchronous event occurs a given number of times |
| 15 | +(typically exactly once or never at all.) This proposal enhances that interface |
| 16 | +to allow arbitrary ranges of event counts so that a test can be written against |
| 17 | +code that may not always fire said event the exact same number of times. |
| 18 | + |
| 19 | +## Motivation |
| 20 | + |
| 21 | +Some tests rely on fixtures or external state that is not perfectly |
| 22 | +deterministic. For example, consider a test that checks that clicking the mouse |
| 23 | +button will generate a `.mouseClicked` event. Such a test might use the |
| 24 | +`confirmation()` interface: |
| 25 | + |
| 26 | +```swift |
| 27 | +await confirmation(expectedCount: 1) { mouseClicked in |
| 28 | + var eventLoop = EventLoop() |
| 29 | + eventLoop.eventHandler = { event in |
| 30 | + if event == .mouseClicked { |
| 31 | + mouseClicked() |
| 32 | + } |
| 33 | + } |
| 34 | + await eventLoop.simulate(.mouseClicked) |
| 35 | +} |
| 36 | +``` |
| 37 | + |
| 38 | +But what happens if the user _actually_ clicks a mouse button while this test is |
| 39 | +running? That might trigger a _second_ `.mouseClicked` event, and then the test |
| 40 | +will fail spuriously. |
| 41 | + |
| 42 | +## Proposed solution |
| 43 | + |
| 44 | +If the test author could instead indicate to Swift Testing that their test will |
| 45 | +generate _one or more_ events, they could avoid spurious failures: |
| 46 | + |
| 47 | +```swift |
| 48 | +await confirmation(expectedCount: 1...) { mouseClicked in |
| 49 | + ... |
| 50 | +} |
| 51 | +``` |
| 52 | + |
| 53 | +With this proposal, we add an overload of `confirmation()` that takes any range |
| 54 | +expression instead of a single integer value (which is still accepted via the |
| 55 | +existing overload.) |
| 56 | + |
| 57 | +## Detailed design |
| 58 | + |
| 59 | +A new overload of `confirmation()` is added: |
| 60 | + |
| 61 | +```swift |
| 62 | +/// Confirm that some event occurs during the invocation of a function. |
| 63 | +/// |
| 64 | +/// - Parameters: |
| 65 | +/// - comment: An optional comment to apply to any issues generated by this |
| 66 | +/// function. |
| 67 | +/// - expectedCount: A range of integers indicating the number of times the |
| 68 | +/// expected event should occur when `body` is invoked. |
| 69 | +/// - isolation: The actor to which `body` is isolated, if any. |
| 70 | +/// - sourceLocation: The source location to which any recorded issues should |
| 71 | +/// be attributed. |
| 72 | +/// - body: The function to invoke. |
| 73 | +/// |
| 74 | +/// - Returns: Whatever is returned by `body`. |
| 75 | +/// |
| 76 | +/// - Throws: Whatever is thrown by `body`. |
| 77 | +/// |
| 78 | +/// Use confirmations to check that an event occurs while a test is running in |
| 79 | +/// complex scenarios where `#expect()` and `#require()` are insufficient. For |
| 80 | +/// example, a confirmation may be useful when an expected event occurs: |
| 81 | +/// |
| 82 | +/// - In a context that cannot be awaited by the calling function such as an |
| 83 | +/// event handler or delegate callback; |
| 84 | +/// - More than once, or never; or |
| 85 | +/// - As a callback that is invoked as part of a larger operation. |
| 86 | +/// |
| 87 | +/// To use a confirmation, pass a closure containing the work to be performed. |
| 88 | +/// The testing library will then pass an instance of ``Confirmation`` to the |
| 89 | +/// closure. Every time the event in question occurs, the closure should call |
| 90 | +/// the confirmation: |
| 91 | +/// |
| 92 | +/// ```swift |
| 93 | +/// let minBuns = 5 |
| 94 | +/// let maxBuns = 10 |
| 95 | +/// await confirmation( |
| 96 | +/// "Baked between \(minBuns) and \(maxBuns) buns", |
| 97 | +/// expectedCount: minBuns ... maxBuns |
| 98 | +/// ) { bunBaked in |
| 99 | +/// foodTruck.eventHandler = { event in |
| 100 | +/// if event == .baked(.cinnamonBun) { |
| 101 | +/// bunBaked() |
| 102 | +/// } |
| 103 | +/// } |
| 104 | +/// await foodTruck.bakeTray(of: .cinnamonBun) |
| 105 | +/// } |
| 106 | +/// ``` |
| 107 | +/// |
| 108 | +/// When the closure returns, the testing library checks if the confirmation's |
| 109 | +/// preconditions have been met, and records an issue if they have not. |
| 110 | +/// |
| 111 | +/// If an exact count is expected, use |
| 112 | +/// ``confirmation(_:expectedCount:isolation:sourceLocation:_:)`` instead. |
| 113 | +public func confirmation<R>( |
| 114 | + _ comment: Comment? = nil, |
| 115 | + expectedCount: some RangeExpression<Int> & Sequence<Int> Sendable, |
| 116 | + isolation: isolated (any Actor)? = #isolation, |
| 117 | + sourceLocation: SourceLocation = #_sourceLocation, |
| 118 | + _ body: (Confirmation) async throws -> sending R |
| 119 | +) async rethrows -> R |
| 120 | +``` |
| 121 | + |
| 122 | +### Ranges without lower bounds |
| 123 | + |
| 124 | +Certain types of range, specifically [`PartialRangeUpTo`](https://developer.apple.com/documentation/swift/partialrangeupto) |
| 125 | +and [`PartialRangeThrough`](https://developer.apple.com/documentation/swift/partialrangethrough), |
| 126 | +may have surprising behavior when used with this new interface because they |
| 127 | +implicitly include `0`. If a test author writes `...10`, do they mean "zero to |
| 128 | +ten" or "one to ten"? The programmatic meaning is the former, but some test |
| 129 | +authors might mean the latter. If an event does not occur, a test using |
| 130 | +`confirmation()` and this `expectedCount` value would pass when the test author |
| 131 | +meant for it to fail. |
| 132 | + |
| 133 | +The unbounded range (`...`) type `UnboundedRange` is effectively useless when |
| 134 | +used with this interface and any use of it here is almost certainly a programmer |
| 135 | +error. |
| 136 | + |
| 137 | +`PartialRangeUpTo` and `PartialRangeThrough` conform to `RangeExpression`, but |
| 138 | +not to `Sequence`, so they will be rejected at compile time. `UnboundedRange` is |
| 139 | +a non-nominal type and will not match either. We will provide unavailable |
| 140 | +overloads of `confirmation()` for these types with messages that explain why |
| 141 | +they are unavailable, e.g.: |
| 142 | + |
| 143 | +```swift |
| 144 | +@available(*, unavailable, message: "Unbounded range '...' has no effect when used with a confirmation.") |
| 145 | +public func confirmation<R>( |
| 146 | + _ comment: Comment? = nil, |
| 147 | + expectedCount: UnboundedRange, |
| 148 | + isolation: isolated (any Actor)? = #isolation, |
| 149 | + sourceLocation: SourceLocation = #_sourceLocation, |
| 150 | + _ body: (Confirmation) async throws -> R |
| 151 | +) async rethrows -> R |
| 152 | +``` |
| 153 | + |
| 154 | +## Source compatibility |
| 155 | + |
| 156 | +This change is additive. Existing tests are unaffected. |
| 157 | + |
| 158 | +Code that refers to `confirmation(_:expectedCount:isolation:sourceLocation:_:)` |
| 159 | +by symbol name may need to add a contextual type to disambiguate the two |
| 160 | +overloads at compile time. |
| 161 | + |
| 162 | +## Integration with supporting tools |
| 163 | + |
| 164 | +The type of the associated value `expected` for the `Issue.Kind` case |
| 165 | +`confirmationMiscounted(actual:expected:)` will change from `Int` to |
| 166 | +`any RangeExpression & Sendable`[^1]. Tools that implement event handlers and |
| 167 | +distinguish between `Issue.Kind` cases are advised not to assume the type of |
| 168 | +this value is `Int`. |
| 169 | + |
| 170 | +## Alternatives considered |
| 171 | + |
| 172 | +- Doing nothing. We have identified real-world use cases for this interface |
| 173 | + including in Swift Testing’s own test target. |
| 174 | +- Allowing the use of any value as the `expectedCount` argument so long as it |
| 175 | + conforms to a protocol `ExpectedCount` (we'd have range types and `Int` |
| 176 | + conform by default.) It was unclear what this sort of flexibility would let |
| 177 | + us do, and posed challenges for encoding and decoding events and issues when |
| 178 | + using the JSON event stream interface. |
| 179 | + |
| 180 | +## Acknowledgments |
| 181 | + |
| 182 | +Thanks to the testing team for their help preparing this pitch! |
| 183 | + |
| 184 | +[^1]: In the future, this type will change to |
| 185 | + `any RangeExpression<Int> & Sendable`. Compiler support is required |
| 186 | + ([96960993](rdar://96960993)). |
0 commit comments