Skip to content

Conversation

@tomerqodo
Copy link

@tomerqodo tomerqodo commented Jan 29, 2026

Benchmark PR from agentic-review-benchmarks#12

Summary by CodeRabbit

  • Tests
    • Improved test infrastructure by replacing test mocks with real value capture methods, enhancing test reliability and accuracy.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link

coderabbitai bot commented Jan 29, 2026

Walkthrough

The changes replace two mock classes (MockWKFrameInfo and MockWKSecurityOrigin) with a new test utility that captures real WKFrameInfo and WKSecurityOrigin objects from an actual WKWebView navigation. Test code is updated to use the real values instead of mocked ones.

Changes

Cohort / File(s) Summary
Mock Removal
BrowserKit/Tests/WebEngineTests/Mock/MockWKFrameInfo.swift, BrowserKit/Tests/WebEngineTests/Mock/MockWKSecurityOrigin.swift
Deleted mock classes that subclassed WKFrameInfo and WKSecurityOrigin with overridable properties and factory methods.
Test Utilities Addition
BrowserKit/Tests/WebEngineTests/Utilities/WebKitTestHelpers.swift
Added new helper utility with captureFrameAndOrigin(for:timeout:) method that creates a real WKWebView, performs navigation, and captures actual frame and security origin values using a navigation delegate.
Test Migration
BrowserKit/Tests/WebEngineTests/WKUIHandlerTests.swift
Updated test to replace MockWKSecurityOrigin and MockWKFrameInfo usage with real values obtained from WebKitTestHelpers, adding guard checks to ensure availability.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 The mocks have hopped away today,
Real values bound in every way,
A web of threads that catch the frame,
True origins—not just a name!
Carrots grow where fakes once lay,
Our tests now feast on real buffet! 🥕

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title describes fixing a test failure on XCode 26.2, which aligns with the main change of replacing mock objects with real WKKit objects to resolve test crashes.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@BrowserKit/Tests/WebEngineTests/WKUIHandlerTests.swift`:
- Around line 38-55: The test times out because expectation is created after
calling wait and never fulfilled; move creation of the XCTestExpectation to
before invoking subject.webView, capture it in the decision handler passed to
subject.webView(requestMediaCapturePermissionFor:initiatedByFrame:type:decisionHandler:),
and call expectation.fulfill() inside the decision handler (the closure
currently referenced as decisionHandler) after asserting
XCTAssertEqual(decision, .prompt); this ensures
WebKitTestHelpers.captureFrameAndOrigin(for:) / subject.webView(...) triggers
the closure which then fulfills the expectation.
🧹 Nitpick comments (2)
BrowserKit/Tests/WebEngineTests/Utilities/WebKitTestHelpers.swift (2)

12-12: Enforce main-thread usage for WKWebView in the helper.

WKWebView APIs are main-thread only; without an explicit main-actor annotation, this helper can be accidentally called off-main later. Consider enforcing this at the API boundary.

✅ Suggested change
-final class WebKitTestHelpers {
+@MainActor
+final class WebKitTestHelpers {

34-48: Handle the wait result explicitly to avoid masking timeouts.

XCTWaiter.wait’s return value is unused. A guard on .completed both removes the unused variable and makes timeout behavior explicit.

✅ Suggested change
-        let waiter = XCTWaiter.wait(for: [expect], timeout: timeout)
-        if let frame = delegate.capturedFrame, let origin = delegate.capturedOrigin {
+        let result = XCTWaiter.wait(for: [expect], timeout: timeout)
+        guard result == .completed else { return nil }
+        if let frame = delegate.capturedFrame, let origin = delegate.capturedOrigin {
             return (frame, origin)
         }
📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 498b08f and 6714f44.

📒 Files selected for processing (4)
  • BrowserKit/Tests/WebEngineTests/Mock/MockWKFrameInfo.swift
  • BrowserKit/Tests/WebEngineTests/Mock/MockWKSecurityOrigin.swift
  • BrowserKit/Tests/WebEngineTests/Utilities/WebKitTestHelpers.swift
  • BrowserKit/Tests/WebEngineTests/WKUIHandlerTests.swift
💤 Files with no reviewable changes (2)
  • BrowserKit/Tests/WebEngineTests/Mock/MockWKSecurityOrigin.swift
  • BrowserKit/Tests/WebEngineTests/Mock/MockWKFrameInfo.swift
🧰 Additional context used
🧬 Code graph analysis (1)
BrowserKit/Tests/WebEngineTests/WKUIHandlerTests.swift (1)
BrowserKit/Tests/WebEngineTests/Utilities/WebKitTestHelpers.swift (2)
  • captureFrameAndOrigin (34-49)
  • webView (20-29)
🔇 Additional comments (2)
BrowserKit/Tests/WebEngineTests/WKUIHandlerTests.swift (2)

59-76: LGTM for this test update.


81-98: LGTM for this test update.

✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.

Comment on lines +38 to 55
guard let (frame, origin) = WebKitTestHelpers.captureFrameAndOrigin(for: URL(string: "https://example.com")!) else {
XCTFail("Could not obtain WKFrameInfo")
return
}

let decisionHandler = { (decision: WKPermissionDecision) in
XCTAssertEqual(decision, .prompt)
expectation.fulfill()
}

let expectation = expectation(description: "Wait for the decision handler to be called")
subject.webView(MockWKWebView(),
requestMediaCapturePermissionFor: MockWKSecurityOrigin.new(nil),
initiatedByFrame: MockWKFrameInfo(),
requestMediaCapturePermissionFor: origin,
initiatedByFrame: frame,
type: .cameraAndMicrophone,
decisionHandler: decisionHandler
)
decisionHandler: decisionHandler)
wait(for: [expectation])
expectation.fulfill()
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

find . -name "WKUIHandlerTests.swift" -type f

Repository: qodo-benchmark/firefox-ios-combined-coderabbit

Length of output: 146


🏁 Script executed:

cat -n ./BrowserKit/Tests/WebEngineTests/WKUIHandlerTests.swift | head -100

Repository: qodo-benchmark/firefox-ios-combined-coderabbit

Length of output: 4546


Fix expectation fulfillment order (test currently times out).

Line 53 calls wait(for:) before line 54 calls expectation.fulfill(), so the wait will never complete and the test will timeout. Move expectation creation before the closure and add fulfillment inside the decision handler.

Suggested fix
-        let decisionHandler = { (decision: WKPermissionDecision) in
-            XCTAssertEqual(decision, .prompt)
-        }
-
-        let expectation = expectation(description: "Wait for the decision handler to be called")
+        let expectation = expectation(description: "Wait for the decision handler to be called")
+        let decisionHandler = { (decision: WKPermissionDecision) in
+            XCTAssertEqual(decision, .prompt)
+            expectation.fulfill()
+        }
         subject.webView(MockWKWebView(),
                         requestMediaCapturePermissionFor: origin,
                         initiatedByFrame: frame,
                         type: .cameraAndMicrophone,
                         decisionHandler: decisionHandler)
         wait(for: [expectation])
-        expectation.fulfill()
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
guard let (frame, origin) = WebKitTestHelpers.captureFrameAndOrigin(for: URL(string: "https://example.com")!) else {
XCTFail("Could not obtain WKFrameInfo")
return
}
let decisionHandler = { (decision: WKPermissionDecision) in
XCTAssertEqual(decision, .prompt)
expectation.fulfill()
}
let expectation = expectation(description: "Wait for the decision handler to be called")
subject.webView(MockWKWebView(),
requestMediaCapturePermissionFor: MockWKSecurityOrigin.new(nil),
initiatedByFrame: MockWKFrameInfo(),
requestMediaCapturePermissionFor: origin,
initiatedByFrame: frame,
type: .cameraAndMicrophone,
decisionHandler: decisionHandler
)
decisionHandler: decisionHandler)
wait(for: [expectation])
expectation.fulfill()
}
let expectation = expectation(description: "Wait for the decision handler to be called")
let decisionHandler = { (decision: WKPermissionDecision) in
XCTAssertEqual(decision, .prompt)
expectation.fulfill()
}
subject.webView(MockWKWebView(),
requestMediaCapturePermissionFor: origin,
initiatedByFrame: frame,
type: .cameraAndMicrophone,
decisionHandler: decisionHandler)
wait(for: [expectation])
🤖 Prompt for AI Agents
In `@BrowserKit/Tests/WebEngineTests/WKUIHandlerTests.swift` around lines 38 - 55,
The test times out because expectation is created after calling wait and never
fulfilled; move creation of the XCTestExpectation to before invoking
subject.webView, capture it in the decision handler passed to
subject.webView(requestMediaCapturePermissionFor:initiatedByFrame:type:decisionHandler:),
and call expectation.fulfill() inside the decision handler (the closure
currently referenced as decisionHandler) after asserting
XCTAssertEqual(decision, .prompt); this ensures
WebKitTestHelpers.captureFrameAndOrigin(for:) / subject.webView(...) triggers
the closure which then fulfills the expectation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants