diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 657cd221..8eec05a7 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -1,6 +1,7 @@ # Summary - [Welcome](welcome.md) + - [Cautions](cautions.md) - [Installation](installation.md) - [Getting started](getting-started.md) - [Using the results](using-results.md) diff --git a/book/src/cautions.md b/book/src/cautions.md new file mode 100644 index 00000000..7e9d9337 --- /dev/null +++ b/book/src/cautions.md @@ -0,0 +1,30 @@ +# Cautions + +cargo-mutants is generally as safe to run as `cargo test`, but there are two important safety considerations: + +## Test side effects + +cargo-mutants introduces machine-generated modifications that simulate bugs. + +If your tests have external side effects (like file operations), these modifications could cause unintended consequences. For example, while cargo-mutants won't add new file deletion calls, it might modify existing deletion paths in your code. A bug in such code could potentially delete unintended directories. + +Most test suites are designed to limit side effects to temporary test resources, even when buggy. However, to minimize risk: + +1. Maintain frequent remote backups +1. Run in an isolated environment (container, CI, or VM) +1. Avoid using production credentials in tests + +## Source tree modifications + +By default, cargo-mutants works on a copy of your source tree and only writes to a `mutants.out` directory. + +However, when using [`--in-place`](in-place.md), it modifies your original source directory. While these changes are normally reverted after testing, sudden interruptions may leave mutations in place. + +When using `--in-place`, either: + +1. Use a dedicated disposable checkout, or +2. Review all diffs carefully before committing + +You can detect mutations by searching for this marker: + + /* ~ changed by cargo-mutants ~ */ diff --git a/book/src/in-place.md b/book/src/in-place.md index 2191b6b8..f0ac98ff 100644 --- a/book/src/in-place.md +++ b/book/src/in-place.md @@ -8,10 +8,19 @@ With the `--in-place` option, it will instead mutate and test your code in the o ## Cautions -If you use `--in-place` then you shouldn't edit the code, commit, or run your own tests while tests are running, because cargo-mutants will be modifying the code at the same time. It's not the default because of the risk that users might accidentally do this. +If you use `--in-place` then you shouldn't edit the code, commit, or run your own tests while tests are running, because cargo-mutants will be modifying the code at the same time. cargo-mutants will try to restore the code to its original state after testing each mutant, but it's possible that it might fail to do so if it's interrupted or panics. +When using `--in-place`, either: + +1. Use a dedicated disposable checkout, or +2. Review all diffs carefully before committing + +You can detect mutations by searching for this marker: + + /* ~ changed by cargo-mutants ~ */ + ## Why test in place? Some situations where `--in-place` might be useful are: @@ -19,4 +28,5 @@ Some situations where `--in-place` might be useful are: * You're running cargo-mutants in CI with a source checkout that exists solely for testing, so it would be a waste of time and space to copy it. * You've previously built the tree into `target` and want to avoid rebuilding it: the Rust toolchain currently doesn't reuse build products after cargo-mutants copies the tree, but it will reuse them with `--in-place`. * The source tree is extremely large, and making a copy would use too much disk space, or take time that you don't want to spend. (In most cases copying the tree takes negligible time compared to running the tests, but if it contains many binary assets it might be significant.) +* Your tree only works when built at a specific filesystem path, or it refers to other directories that cargo-mutants doesn't understand how to copy. * You're investigating or debugging a problem where the tests don't pass in a copy of the tree. (Please report this as a bug if you can describe how to reproduce it.) diff --git a/book/src/limitations.md b/book/src/limitations.md index 9499b3d7..312c4df9 100644 --- a/book/src/limitations.md +++ b/book/src/limitations.md @@ -54,17 +54,3 @@ cargo-mutants contains two main categories of code, which are mostly independent The main precondition for supporting Bazel is a realistic test case: preferably an open source Rust tree built with Bazel, or at least a contributor with a Bazel-based Rust tree who is willing to help test and debug and to produce some test cases. (See for more discussion.) - -## Caution on side effects - -cargo-mutants builds and runs code with machine-generated modifications. This is -generally fine, but if the code under test has side effects such as writing or -deleting files, running it with mutations might conceivably have unexpected -effects, such as deleting the wrong files, in the same way that a bug might. - -If you're concerned about this, run cargo-mutants in a container or virtual -machine. - -cargo-mutants never modifies the original source tree, other than writing a -`mutants.out` directory, and that can be sent elsewhere with the `--output` -option. All mutations are applied and tested in a copy of the source tree.