Skip to content

Commit 8847750

Browse files
committed
chore: add an intermittent test failure policy
1 parent a60e336 commit 8847750

File tree

1 file changed

+17
-0
lines changed

1 file changed

+17
-0
lines changed

CONTRIBUTING.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,23 @@ cargo insta test
5757
cargo insta review
5858
```
5959

60+
### Intermittent failure policy
61+
62+
While we strive to add test coverage for as many features as we can, it sometimes happens that the
63+
tests will be intermittently failing in CI (such tests are sometimes called "flaky"). This can be
64+
caused by race conditions of all sorts, either in the test code itself, but sometimes in the
65+
underlying feature being tested too, and as such, it requires some investigation, usually from the
66+
original author of the test.
67+
68+
Whenever such an intermittent failure happens, we try to open an issue to track the failures,
69+
adding the
70+
[`intermittent-failure`](https://github.com/matrix-org/matrix-rust-sdk/issues?q=is%3Aissue%20state%3Aopen%20label%3Aintermittent-failure)
71+
label to it, and commenting with links to CI runs where the failure happened.
72+
73+
If a test has been intermittently failing for **two weeks** or more, and no one is actively working
74+
on fixing it, then we might decide to mark the test as `ignored` until it is fixed, to not cause
75+
unrelated failures in other contributors' pull requests and pushes.
76+
6077
## Pull requests
6178

6279
Ideally, a PR should have a *proper title*, with *atomic logical commits*, and

0 commit comments

Comments
 (0)