Respond to review feedback

Author: mhauru

developers/contributing/index.qmd

@@ -54,7 +54,7 @@ Or otherwise, set the global `ARGS` variable, and call `include("test/runtests.j
 We merge all code changes through pull requests on GitHub.
 To make a contribution to one of the Turing packages, fork it on GitHub, start a new branch on your fork, and add commits to it.
 Once you're done, open a pull request to the main repository under [TuringLang](https://github.com/TuringLang).
-Someone from the dev team will review your code (if they don't, ping `@TuringLang/maintainers` in a comment to get their attention) and check that the continuous integration tests pass.
+Someone from the dev team will review your code (if they don't, ping `@TuringLang/maintainers` in a comment to get their attention) and check that the continuous integration tests pass (with some allowed exceptions, see below).
 If all looks good, we'll merge your PR with much joy and gratitude If not, we'll help you fix it and then merge it with much joy and gratitude
 
 Everything in this section about pull requests and branches applies to the Turing.jl and DynamicPPL.jl repositories.
@@ -63,7 +63,7 @@ As of August 2025 we are slowly moving towards having all repos do the full proc
 
 #### Branches
 
-Like Julia packages generally, Turing follows [semantic versioning](https://semver.org/).
+Like Julia packages generally, Turing.jl follows [semantic versioning](https://semver.org/).
 Because of this, we have two persistently alive branches in our repository: `main` and `breaking`.
 All code that gets released as a new version of Turing gets merged into `main`, and a release is made from there.
 However, any breaking changes should first be merged into `breaking`.
@@ -80,7 +80,7 @@ Meanwhile, breaking changes can be developed and merged into `breaking`, which i
 Multiple breaking changes may be accumulated into `breaking`, before finally the `breaking`-to-`main` merge is done, and 0.20.0 is released.
 On `breaking` the version number should then immediately be bumped to 0.21.
 
-We do not generally bother doing backports of bug fixes, though we may consider them in special circumstances.
+We do not generally back port of bug fixes, although we may consider doing so in special circumstances.
 
 #### Change history
 
@@ -90,6 +90,23 @@ Any major new features should also be described in `HISTORY.md`, as may any othe
 Bug fixes generally don't need an entry in `HISTORY.md`.
 Any new breaking release must have an entry in `HISTORY.md`, entries for non-breaking releases are optional.
 
+#### Continuous integration (CI) tests
+
+We generally run the whole test suite of each repository in a GitHub action, typically for a few different versions of Julia, including the earliest supported version and the latest stable release.
+On some repositories we also run a few other checks in CI, such as code formatting and simple benchmarks.
+Generally all tests except those run on a prerelease version of Julia (e.g. a release candidate of an upcoming Julia release), and all code formatting checks, should pass before merging a PR.
+Exceptions can be made if the cause of the failure is known and unrelated to the PR.
+CI checks other than tests and formatting serve various purposes, and some of them can be allowed to fail.
+Some examples are
+
+  - Anything running on a prerelease of Julia. These inform us of trouble ahead when that prerelease becomes an actual release, but don't require fixing for a PR to be merged.
+  - Any code coverage checks. Code coverate numbers can be helpful in catching missing tests or cases where the tests don't test what they are intended to. However, we do not insist on any particular coverage figures, since they are not a very good metric of a test suite's extensiveness.
+  - The benchmarks on DynamicPPL repo. These should be investigated to understand why they fail. If the reason is a bug in the PR, an actual test should be added to the test suite to catch it. However, sometimes they fail for unrelated reasons.
+  - Checks against some particular automatic differentiation backends. These are informative, telling us which backends have issues with which features, but often their breakage is beyond our control and we let it pass.
+  - The CI check in the `docs` repo for whether the docs are built with the latest Turing.jl release. This test failing is a reminder that we should make a PR to update to the latest version, but does not need fixing when working on a PR that makes unrelated changes to the documentation.
+
+If you are ever unsure whether some CI check needs to pass, or if the reason why one is failing is mysterious or seems unrelated to the PR, ask a maintainer and they'll help you out.
+
 #### Please make mistakes
 
 Getting pull requests from outside the core developer team is one of the greatest joys of open source maintenance, and Turing's community of contributors is its greatest asset.