Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ import Storage

@testable import Client

@MainActor
final class StoriesFeedDiffableDataSourceTests: XCTestCase {
var collectionView: UICollectionView?
var diffableDataSource: StoriesFeedDiffableDataSource?

override func setUpWithError() throws {
try super.setUpWithError()

override func setUp() async throws {
try await super.setUp()
collectionView = UICollectionView(frame: .zero, collectionViewLayout: UICollectionViewFlowLayout())
let collectionView = try XCTUnwrap(collectionView)
diffableDataSource = StoriesFeedDiffableDataSource(
Expand All @@ -25,11 +25,11 @@ final class StoriesFeedDiffableDataSourceTests: XCTestCase {
DependencyHelperMock().bootstrapDependencies()
}

override func tearDown() {
override func tearDown() async throws {
try await super.tearDown()
diffableDataSource = nil
collectionView = nil
DependencyHelperMock().reset()
super.tearDown()
}
Comment on lines +28 to 33
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🌐 Web query:

XCTestCase async tearDown exception handling best practices

💡 Result:

Best practices for XCTestCase async tearDown + exception/error handling

1) In tearDown() async throws, only do work that is actually awaited

If you schedule work with DispatchQueue.async/asyncAfter, XCTest won’t wait for it (because those APIs aren’t async), so cleanup may never run. Prefer await-based APIs (or wrap callback-based APIs with continuations). (stackoverflow.com)

override func tearDown() async throws {
  try await apiClient.deleteTestData()
  await super.tearDown()
}

2) Prefer teardown blocks (addTeardownBlock) for per-test cleanup (especially when partially-initialized)

Register cleanup right after you allocate/create a resource, so it gets cleaned up even if the test fails midway.

Notes from practice/documentation summaries:

  • teardown blocks run after the test method and before the tearDown methods
  • the block can await and throw
  • you can register teardown blocks from the test or setUp, but not from tearDown (useyourloaf.com)
func testSomething() async throws {
  let id = try await apiClient.createThing()

  addTeardownBlock {
    try await apiClient.deleteThing(id: id)
  }

  // ... assertions ...
}

3) Decide whether teardown errors should fail the test or be best-effort

  • If you want cleanup failures to fail the test, let the error propagate by using tearDownWithError / tearDown() async throws. (Throwing from these methods is treated as a test failure.) (useyourloaf.com)
  • If cleanup is best-effort (common for “try to delete remote test data”), catch and record, or just swallow—otherwise you can end up with many “failed tests” caused only by cleanup.

4) Avoid continueAfterFailure = false with async tests / async setUp / async tearDown

continueAfterFailure = false is implemented via Objective‑C exceptions; exceptions through Swift async frames are problematic/undefined, and newer Xcode versions changed behavior to avoid reliability issues. Practically: keep continueAfterFailure at the default true, and use throws/early-return patterns to stop the test. (gist.github.com)

5) Don’t start “detached” tasks in teardown unless you intentionally don’t care if they finish

If you do Task { ... } in teardown and don’t await it, the test can end before it completes, and the task may be cancelled or never observed. Prefer structured concurrency (await the work in teardown / teardown block). (stackoverflow.com)

Citations:


🌐 Web query:

Swift XCTest tearDown super call order cleanup

💡 Result:

In XCTest, the conventional (and safest) pattern is:

  • setUp…: call super first, then do your per-test setup.
  • tearDown…: do your cleanup first, then call super last. This matches Apple’s long-standing example (tearDown ends with [super tearDown]). [1]

Why this order matters / what runs when:

  • Any addTeardownBlock { … } blocks run after the test method ends and before the test case’s teardown methods. Multiple teardown blocks run LIFO (last registered, first executed). [2]
  • For a given test, XCTest runs setup/teardown hooks in a defined sequence (showing the common overloads):
    • Setup: setUp() async throwssetUpWithError()setUp()
    • Teardown: tearDown()tearDownWithError()tearDown() async throws (and teardown blocks occur before these). [3]

Do you need to call super?

  • For a direct subclass of XCTestCase, the base implementations are effectively “template methods” and are often empty, so omitting super usually doesn’t change behavior today—but calling super keeps things correct if you later introduce a shared test base class or a framework superclass adds behavior. [4]

Typical Swift shape

override func setUp() throws {
  try super.setUp()
  // create SUT, fixtures
}

override func tearDown() throws {
  // nil out SUT, close files, cancel tasks, etc.
  try super.tearDown()
}

Sources: Apple archived testing guide example [1], teardown-block ordering notes [2], lifecycle ordering summary [3], discussion of why super calls are optional-but-wise [4].

[1] https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/testing_with_xcode/chapters/04-writing_tests.html
[2] https://useyourloaf.com/blog/unit-test-setup-and-teardown/
[3] https://useyourloaf.com/blog/unit-test-setup-and-teardown/
[4] https://stackoverflow.com/questions/53448538/xcode-10-and-super-teardown


Move cleanup before super.tearDown() to ensure it runs even if super throws.

XCTest's conventional pattern places cleanup before the super call in tearDown methods. If super.tearDown() throws an exception, the subsequent cleanup (nil assignments and DependencyHelperMock().reset()) won't execute, leaking state into later tests.

♻️ Proposed fix
 override func tearDown() async throws {
-    try await super.tearDown()
     diffableDataSource = nil
     collectionView = nil
     DependencyHelperMock().reset()
+    try await super.tearDown()
 }
🤖 Prompt for AI Agents
In
`@firefox-ios/firefox-ios-tests/Tests/ClientTests/Frontend/StoriesFeed/StoriesFeedDiffableDataSourceTests.swift`
around lines 28 - 33, Move the cleanup (nil assignments and mock reset) to run
before calling super. In the tearDown() implementation, first set
diffableDataSource = nil, collectionView = nil and call
DependencyHelperMock().reset(), then call try await super.tearDown() so cleanup
always executes even if super throws; update the tearDown() method accordingly.


func test_updateSnapshot_initialSnapshotHasNoData() throws {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,20 @@ class SyncContentSettingsViewControllerTests: XCTestCase {
var syncContentSettingsVC: SyncContentSettingsViewController?
let windowUUID: WindowUUID = .XCTestDefaultUUID

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
DependencyHelperMock().bootstrapDependencies()
profile = MockProfile()
LegacyFeatureFlagsManager.shared.initializeDeveloperFeatures(with: profile)
syncContentSettingsVC = SyncContentSettingsViewController(windowUUID: windowUUID)
syncContentSettingsVC?.profile = profile
}

override func tearDown() {
override func tearDown() async throws {
DependencyHelperMock().reset()
profile = nil
syncContentSettingsVC = nil
super.tearDown()
try await super.tearDown()
}

func test_syncContentSettingsViewController_generateSettingsCount() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,19 @@ class FxAWebViewModelTests: XCTestCase {
var viewModel: FxAWebViewModel!
var deeplinkParams: FxALaunchParams!

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
deeplinkParams = FxALaunchParams(entrypoint: .browserMenu, query: ["test_key": "test_value"])
viewModel = FxAWebViewModel(pageType: .settingsPage,
profile: MockProfile(),
deepLinkParams: deeplinkParams,
telemetry: FxAWebViewTelemetry(telemetryWrapper: MockTelemetryWrapper()))
}

override func tearDown() {
override func tearDown() async throws {
deeplinkParams = nil
viewModel = nil
super.tearDown()
try await super.tearDown()
}

func testCreateOutputURLWithValidFileNameAndExtension() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ class NotificationManagerTests: XCTestCase {
}

func testRequestAuthorization() {
notificationManager.requestAuthorization { (granted, error) in
XCTAssertTrue(granted)
XCTAssertTrue(self.center.requestAuthorizationWasCalled)
notificationManager.requestAuthorization { [center] (granted, error) in
assert(granted, "Authorization should be granted")
assert(center?.requestAuthorizationWasCalled ?? false, "requestAuthorization should be called")
Comment on lines +26 to +28
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cat -n firefox-ios/firefox-ios-tests/Tests/ClientTests/Helpers/NotificationManagerTests.swift

Repository: qodo-benchmark/firefox-ios-combined-coderabbit

Length of output: 5301


🌐 Web query:

Swift 6 XCTest assertions @sendable completion handlers best practices

💡 Result:

Best practices (Swift 6 + strict concurrency) for XCTest assertions with @Sendable completion handlers

1) Prefer async tests + await fulfillment(of:timeout:) over blocking waits

Blocking APIs like wait(for:timeout:) / waitForExpectations(timeout:) can deadlock when mixed with async work; Apple introduced the concurrency-safe fulfillment(of:timeout:enforceOrder:) specifically for this scenario. Use it in async tests. [1], [2]

func testFetch() async throws {
  let exp = expectation(description: "callback")
  api.fetch { exp.fulfill() }
  await fulfillment(of: [exp], timeout: 1.0)
}

2) Don’t put XCTest assertions in callbacks that may run concurrently unless you control isolation

If the completion can run on any executor/thread, treat it as concurrent:

  • Make the completion @Sendable (or accept a sending closure—see #4).
  • Only capture Sendable state (or make captured state Sendable/immutable), or hop to an actor before touching isolated/mutable state. This mirrors the general Swift 6 rule that @Sendable closures can’t safely capture non-Sendable values. [3]
api.fetch { `@Sendable` result in
  Task { `@MainActor` in
    XCTAssertEqual(result.value, 42)
  }
}

3) For “UI-style” completions that are guaranteed to run on the main thread, model that explicitly

If a callback is logically “main-actor-only”, prefer typing it as @MainActor (and in practice you may still see @Sendable involved depending on bridging/heuristics). One pragmatic pattern is @Sendable @MainActor`` for the closure type and then call it from a main-actor context. [4]

typealias Completion = `@Sendable` `@MainActor` () -> Void

4) If you own an API that takes a completion handler, consider sending instead of @Sendable

Swift Evolution added sending to reduce caller friction vs requiring @Sendable everywhere (especially for “good old completion blocks”). That said, there have been Swift 6-era rough edges when passing sending closures from actor-isolated contexts (notably @MainActor). [5], [6]

func doWork(completion: sending `@escaping` () -> Void) { ... }

5) Be aware XCTest’s async bridging can itself introduce @Sendable completion handlers

XCTest discovers async tests via an Objective-C thunk that takes a @Sendable completion handler; changes in actor isolation/default isolation can affect how XCTest sees/runs your tests. If you hit odd isolation/compiler issues, check your test target’s default actor isolation and where you apply @MainActor. [7]


Sources

[1] Swift Forums: deadlock + “use fulfillment(of:) instead” release note quote (Xcode 14.3+) — https://forums.swift.org/t/mixing-async-await-code-with-waitforexpectations-in-tests-causes-hang/62773
[2] Apple docs: XCTestCase.fulfillment(of:timeout:enforceOrder:)https://developer.apple.com/documentation/xctest/xctestcase/fulfillment%28of%3Atimeout%3Aenforceorder%3A%29
[3] Stack Overflow: URLSession completion is @Sendable and triggers Swift 6 sendability errors — https://stackoverflow.com/questions/79205410/resolving-concurrency-sendable-errors
[4] Jesse Squires: @Sendable @MainActor`` completion pattern; notes Swift 6 improvements via SE-0434 — https://www.jessesquires.com/blog/2024/06/05/swift-concurrency-non-sendable-closures/
[5] Swift Forums: SE-0430 acceptance announcement (sending) — https://forums.swift.org/t/accepted-with-modifications-se-0430-second-review-sendable-parameter-and-result-values/71850
[6] Swift Forums: “friends don’t let friends declare good old completion blocks as `@Sendable`. They should be `sending`” + `@MainActor` caveat — https://forums.swift.org/t/relaxing-sendable-constraint-to-sending-closures-creates-undesired-warnings-errors-in-swift-6-language-mode/72367
[7] Swift Forums: XCTest async test thunk uses `@Sendable` completion handler; actor-isolation pitfalls — https://forums.swift.org/t/xctestcase-compiler-error-with-swift-6-2-default-actor-isolation-mainactor/83418


Test doesn't wait for completion handler to execute, making assertions invisible.

The testRequestAuthorization() method returns immediately after calling the completion-based API. The assertions inside the handler may never run or fail silently, resulting in a false-positive test. Additionally, assert() is stripped in Release builds and won't surface as XCTest failures.

Convert to an async test with an expectation to ensure the completion handler is awaited and assertions run reliably on the main actor:

Recommended fix
-    func testRequestAuthorization() {
+    func testRequestAuthorization() async {
+        let exp = expectation(description: "requestAuthorization")
         notificationManager.requestAuthorization { [center] (granted, error) in
-            assert(granted, "Authorization should be granted")
-            assert(center?.requestAuthorizationWasCalled ?? false, "requestAuthorization should be called")
+            Task { `@MainActor` in
+                XCTAssertTrue(granted, "Authorization should be granted")
+                XCTAssertTrue(center?.requestAuthorizationWasCalled ?? false, "requestAuthorization should be called")
+                exp.fulfill()
+            }
         }
+        await fulfillment(of: [exp], timeout: 1)
     }
🤖 Prompt for AI Agents
In
`@firefox-ios/firefox-ios-tests/Tests/ClientTests/Helpers/NotificationManagerTests.swift`
around lines 26 - 28, The testRequestAuthorization() currently calls
notificationManager.requestAuthorization and uses assert() inside the completion
handler which may not run before the test exits and is stripped in Release;
convert this to an XCTest expectation or an async test so the test waits for the
completion, and replace assert(...) with XCTest assertions (e.g.
XCTAssertTrue/granted and XCTAssertTrue(center?.requestAuthorizationWasCalled ??
false) or XCTAssertNotNil for center) executed on the main actor; specifically
update the code that calls notificationManager.requestAuthorization { [center]
(granted, error) in ... } to fulfill an expectation (or await an async API)
inside that closure and then waitForExpectations or mark the test async so
assertions inside the completion are reliably executed.

}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ class RatingPromptManagerTests: XCTestCase {
var crashTracker: MockCrashTracker!
var subject: RatingPromptManager!

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()

prefs = MockProfilePrefs()
logger = CrashingMockLogger()
Expand All @@ -33,7 +33,8 @@ class RatingPromptManagerTests: XCTestCase {
userDefaults: userDefaults)
}

override func tearDown() {
override func tearDown() async throws {
try await super.tearDown()
prefs.clearAll()
subject.reset()
prefs = nil
Expand All @@ -42,8 +43,6 @@ class RatingPromptManagerTests: XCTestCase {
userDefaults = nil
crashTracker = nil
subject = nil

super.tearDown()
}

func testShouldShowPrompt_forceShow() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,26 @@
import XCTest
@testable import Client

@MainActor
final class EditBookmarkDataSourceTests: XCTestCase {
private let folders = [
Folder(title: "Parent", guid: "ParentFolder", indentation: 0),
Folder(title: "Child", guid: "ChildFolder", indentation: 1)
]
private var tableView: UITableView!

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
tableView = UITableView()
let window = UIWindow(frame: UIScreen.main.bounds)
window.rootViewController = UIViewController()
window.rootViewController?.view.addSubview(tableView)
window.makeKeyAndVisible()
}

override func tearDown() {
override func tearDown() async throws {
tableView = nil
super.tearDown()
try await super.tearDown()
}

func testOnSnapshotUpdate() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ import Shared

@testable import Client

@MainActor
class EditBookmarkViewModelTests: XCTestCase {
let folder = MockFxBookmarkNode(type: .folder,
guid: "1235",
Expand All @@ -31,18 +30,18 @@ class EditBookmarkViewModelTests: XCTestCase {
var bookmarksSaver: MockBookmarksSaver!
var profile: MockProfile!

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
folderFetcher = MockFolderHierarchyFetcher()
bookmarksSaver = MockBookmarksSaver()
profile = MockProfile()
}

override func tearDown() {
override func tearDown() async throws {
folderFetcher = nil
bookmarksSaver = nil
profile = nil
super.tearDown()
try await super.tearDown()
}

func testInit() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@ import Common
class HistoryPanelTests: XCTestCase {
let windowUUID: WindowUUID = .XCTestDefaultUUID
private var notificationCenter: MockNotificationCenter!
override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
LegacyFeatureFlagsManager.shared.initializeDeveloperFeatures(with: MockProfile())
DependencyHelperMock().bootstrapDependencies()
notificationCenter = MockNotificationCenter()
}

override func tearDown() {
super.tearDown()
override func tearDown() async throws {
DependencyHelperMock().reset()
notificationCenter = nil
try await super.tearDown()
}

func testHistoryButtons() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,16 @@ final class MainMenuConfigurationUtilityTests: XCTestCase {
var configUtility: MainMenuConfigurationUtility!
let windowUUID: WindowUUID = .XCTestDefaultUUID

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
DependencyHelperMock().bootstrapDependencies()
configUtility = MainMenuConfigurationUtility()
}

override func tearDown() {
override func tearDown() async throws {
DependencyHelperMock().reset()
configUtility = nil
super.tearDown()
try await super.tearDown()
}

func testGenerateMenuElements_returnsHomepageSections_whenIsHomepageTrue() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,15 @@ import XCTest
final class MainMenuCoordinatorTests: XCTestCase {
private var mockRouter: MockRouter!

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
DependencyHelperMock().bootstrapDependencies()
mockRouter = MockRouter(navigationController: MockNavigationController())
}

override func tearDown() {
override func tearDown() async throws {
DependencyHelperMock().reset()
super.tearDown()
try await super.tearDown()
}

func testInitialState() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,18 @@ final class MainMenuMiddlewareTests: XCTestCase, StoreTestUtility {
var mockGleanWrapper: MockGleanWrapper!
var mockStore: MockStoreForMiddleware<AppState>!

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
mockGleanWrapper = MockGleanWrapper()
DependencyHelperMock().bootstrapDependencies()
setupStore()
}

override func tearDown() {
override func tearDown() async throws {
DependencyHelperMock().reset()
mockGleanWrapper = nil
resetStore()
super.tearDown()
try await super.tearDown()
}

func test_tapNavigateToDestination_findInPageAction_sendTelemetryData() throws {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import Common

@testable import Client

@MainActor
final class MainMenuViewControllerTests: XCTestCase {
let windowUUID: WindowUUID = .XCTestDefaultUUID

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,16 @@ final class MicrosurveyCoordinatorTests: XCTestCase {
private var mockRouter: MockRouter!
private var mockTabManager: MockTabManager!

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
DependencyHelperMock().bootstrapDependencies()
mockRouter = MockRouter(navigationController: MockNavigationController())
mockTabManager = MockTabManager()
}

override func tearDown() {
override func tearDown() async throws {
DependencyHelperMock().reset()
super.tearDown()
try await super.tearDown()
}

func testInitialState() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,18 @@ final class MicrosurveyMiddlewareIntegrationTests: XCTestCase, StoreTestUtility
var mockStore: MockStoreForMiddleware<AppState>!
var mockMicrosurveyTelemetry: MockMicrosurveyTelemetry!

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
mockMicrosurveyTelemetry = MockMicrosurveyTelemetry()
DependencyHelperMock().bootstrapDependencies()
setupStore()
}

override func tearDown() {
override func tearDown() async throws {
mockMicrosurveyTelemetry = nil
DependencyHelperMock().reset()
resetStore()
super.tearDown()
try await super.tearDown()
}

func testDismissSurveyAction() throws {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import Common

@testable import Client

@MainActor
final class MicrosurveyViewControllerTests: XCTestCase {
let windowUUID: WindowUUID = .XCTestDefaultUUID

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ class MockNavigationAction: WKNavigationAction {
}

// MARK: - MockURLAuthenticationChallengeSender
class MockURLAuthenticationChallengeSender: NSObject, URLAuthenticationChallengeSender {
final class MockURLAuthenticationChallengeSender: NSObject, URLAuthenticationChallengeSender {
func use(_ credential: URLCredential, for challenge: URLAuthenticationChallenge) {}

func continueWithoutCredential(for challenge: URLAuthenticationChallenge) {}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ final class MockTabQueue: TabQueue, @unchecked Sendable {
return succeed()
}

func getQueuedTabs(completion: @escaping @MainActor ([ShareItem]) -> Void) {
func getQueuedTabs(completion: @MainActor @Sendable @escaping ([ShareItem]) -> Void) {
Task { @MainActor in
completion(queuedTabs)
getQueuedTabsCalled += 1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import Foundation
import UserNotifications
@testable import Client

class MockUserNotificationCenter: UserNotificationCenterProtocol {
class MockUserNotificationCenter: UserNotificationCenterProtocol, @unchecked Sendable {
var pendingRequests = [UNNotificationRequest]()

var getSettingsWasCalled = false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import Common

@testable import Client

@MainActor
final class NativeErrorPageViewControllerTests: XCTestCase {
let windowUUID: WindowUUID = .XCTestDefaultUUID

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,15 @@ import XCTest
class IntroViewControllerTests: XCTestCase {
var mockNotificationCenter: MockNotificationCenter!

override func setUp() {
super.setUp()
override func setUp() async throws {
try await super.setUp()
DependencyHelperMock().bootstrapDependencies()
mockNotificationCenter = MockNotificationCenter()
}

override func tearDown() {
super.tearDown()
override func tearDown() async throws {
mockNotificationCenter = nil
try await super.tearDown()
}

// Temp. Disabled: https://mozilla-hub.atlassian.net/browse/FXIOS-7505
Expand Down
Loading