Skip to content

Commit 0875d10

Browse files
committed
feat: mutants docs - time related outcomes
1 parent 9b54fcb commit 0875d10

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

docs/ci-release.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,14 @@ Since mutation testing is directly correlated to the written tests, there are sl
236236

237237
Once all the jobs have finished testing mutants, the last job collects all the tested mutations from the previous jobs, combines them and outputs them to the `Summary` section of the workflow, at the bottom of the page. There, you can find all mutants on categories, with links to the function they tested, and a short description on how to fix the issue. The PR should only be approved/merged after all the mutants tested are in the `Caught` category.
238238

239+
### Time required to run the workflow based on mutants outcome and packages' size
240+
241+
- Small packages typically completed in under 30 minutes, aided by the use of shards.
242+
- Large packages like stackslib and stacks-node initially required about 20-25 minutes for build and test processes.
243+
- Each "missed" and "caught" mutant took approximately 15 minutes. Using shards, this meant about 50-55 minutes for processing around 32 mutants (10-16 functions modified). Every additional 8 mutants added another 15 minutes to the runtime.
244+
- "Unviable" mutants, which are functions lacking a Default implementation for their returned struct type, took less than a minute each.
245+
- "Timeout" mutants typically required more time. However, these should be marked to be skipped (by adding a skip flag to their header) since they indicate functions unable to proceed in their test workflow with mutated values, as opposed to the original implementations.
246+
239247
File:
240248

241249
- [PR Differences Mutants](../.github/workflows/pr-differences-mutants.yml)

0 commit comments

Comments
 (0)