Parallelise tests #3654
Closed
Parallelise tests #3654
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 3de7da22b1926aa96292be600e12d9146d9e7a39.
finestructure
force-pushed
the
parallelise-tests
branch
from
fdfb186 to
caab8cb
Compare
|
Ok, we will definitely have to move to Swift Testing for this to work. I've set up two simple tests that increment an int and sleep for 2s. It's clear that with XCTest these are all independent counts, they all show a count of (I've made the tests to fail so the output is logged to the console. Otherwise it doesn't print anything.) With Swift Testing this behaves as expected: |
…wift Testing does" This reverts commit caab8cb.
|
Closing this for now, follow-up work is in progress. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This works in principle but currently has major drawbacks.
Locally, the tests run a lot slower, because each test spins up its own PG docker container (and then sets up the schema). The serial tests instead re-use the container and only reset the schema, which is a lot faster.
We can fix this (and I've done the same in the past in a python project) by spinning up a pool of PG containers at the start of the test and then scheduling tests against that pool just like we do serially. We just need to mutex each PG instance.
However, that's probably not going to be possible with XCTest based testing, because from initial testing it looks like each test is actually launched as a separate process. I.e. unless we manage the mutex in the file system or somewhere else centrally we won't be able to record which test is using which db. I was planning to do this simply via an actor or a
Mutexguarded entity but each process would have their own copy.I believe the new Swift Testing does not have this problem but I'll run a test with both to see which one would work.
Unfortunately that may mean that we will need to move the tests to Swift Testing to be able to run them in parallel. However, thanks to
swift-testing-revolutionarythis isn't such a huge deal actually. I've done this for theAnalyzerTestssuite and it is mostly an automatic conversion with some touch up.