Skip to content

Conversation

finestructure
Copy link
Member

@finestructure finestructure commented Feb 13, 2025

This converts AnalyzeErrorTests to Swift Testing.

It was a quite a bit more involved than expected, because override func invokeTest does not (yet) have an equivalent in Swift Testing. This should be possible with Swift 6.1.

@cla-bot cla-bot bot added the cla-signed label Feb 13, 2025
@finestructure finestructure force-pushed the issue-3655-swift-testing-part-2 branch from 8cf5e12 to b259c21 Compare February 14, 2025 08:12
@finestructure finestructure marked this pull request as ready for review February 14, 2025 08:38
@finestructure
Copy link
Member Author

Not sure what's going on but the CI keeps failing. All tests pass locally but I can reproduce it on Linux in a dev container. The weird thing is that there's no test failure. It's just a script failure:

...
Test Suite 'WebpageSnapshotTests' passed at 2025-02-14 11:04:04.684
Executed 46 tests, with 0 failures (0 unexpected) in 5.162 (5.162) seconds
Test Suite S3StoreTests started
    ✔ test_Key_objectUrl (0.0 seconds)
    ✔ test_Key_path (0.0 seconds)
    ✔ test_Key_s3Uri (0.0 seconds)
Test Suite 'S3StoreTests' passed at 2025-02-14 11:04:04.685
Executed 3 tests, with 0 failures (0 unexpected) in 0.001 (0.001) seconds
Test Suite 'debug.xctest' passed at 2025-02-14 11:04:04.685
Executed 791 tests, with 20 tests skipped and 0 failures (0 unexpected) in 104.456 (104.456) seconds
Test Suite 'All tests' passed at 2025-02-14 11:04:04.685
Executed 791 tests, with 20 tests skipped and 0 failures (0 unexpected) in 104.456 (104.456) seconds
make: *** [Makefile:37: test] Error 1

Investigating.

@finestructure
Copy link
Member Author

We need to drop xcbeautify. It's eating Swift Testing's output, plus Swift Testing has better test reporting, making it pretty much obsolete. The only downside is that until all tests are converted, we'll have the spammier XCTest reporting and no GH test failure reporting.

@finestructure
Copy link
Member Author

finestructure commented Feb 14, 2025

Actually, we're way behind the latest release of 2.25.0 and there seems to be an update that handles Swift Testing output. Testing.

@finestructure
Copy link
Member Author

D'oh, Swift Testing is running the tests in parallel by default. That's why this test conversion (the first one that's using the db) is failing in CI. My Xcode scheme was set to running them sequentially, that's why they passed there.

I'll set them to run --no-parallel until we've converted them all.

@finestructure finestructure merged commit defa126 into main Feb 15, 2025
6 checks passed
@finestructure finestructure deleted the issue-3655-swift-testing-part-2 branch February 15, 2025 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants