11# Bisecting Regressions
22
33The [ ` cargo-bisect-rustc ` ] tool makes it super easy to find exactly when
4- behavior has changed in rustc. It automatically downloads rustc artifacts and
5- tests them against a project you provide until it finds the regression. To
6- install:
4+ behavior has regressed in rustc. It automatically downloads rustc
5+ artifacts and tests them against a project you provide until it finds
6+ the regression.
7+
8+ To install the tool run:
79
810``` sh
9- git clone https://github.com/rust-lang-nursery/cargo-bisect-rustc.git
10- RUST_SRC_REPO=/path/to/rust cargo install --path=cargo-bisect-rustc
11+ cargo install cargo-bisect-rustc
12+ ```
13+
14+ or (to avoid cloning rustc)
15+
16+ ```
17+ RUST_SRC_REPO=/path/to/rust cargo install cargo-bisect-rustc
1118```
1219
1320The ` RUST_SRC_REPO ` should be a path to a git clone of the rust repo. If you
1421don't specify it, it will look in the current directory for ` rust.git ` or
1522check it out automatically if it's not there (only necessary if doing git hash
1623bisections).
1724
25+ First, if you have a nightly version of the compiler already installed
26+ as the default toolchain and you don't pass an end flag, the tool is
27+ going to assume that that's the version that has regressed and use it as
28+ the "bad" version. Otherwise would use that start point or just use the
29+ latest nightly.
30+ If you have provided a start flag it would use that as the "good"
31+ version, otherwise is going to search for a good one backwards.
32+
33+ Once the tool has an start point (good version) and end point (bad
34+ version), is going to do the bisect to find the regressed nightly. Once
35+ it finds the regressed nightly is going to look between that nightly and
36+ the one of the day before for the specific commit that has the
37+ regression and report everything back to the user.
38+
39+ So, there's a bunch of ways to run the tool, the easiest one is to
40+ allow the tool to do the job and just run ` cargo bisect-rustc ` on the
41+ project and let the tool figure things out.
42+
43+ Note: you can also pass ` RUST_SRC_REPO ` env var, should be a path to a
44+ git clone of the rust repo. If you don't specify it, it will look in the
45+ current directory for ` rust.git ` or check it out automatically if it's
46+ not there (only necessary if doing git hash bisections).
47+
48+ ## Finding a regression
49+
1850Create a cargo project that demonstrates the regression. Let's use
1951[ issue #53157 ] as an example:
2052
@@ -38,20 +70,16 @@ fn main() {
3870}
3971```
4072
41- ## Regressing to a nightly
73+ Then run ` cargo bisect-rustc --end=2018-08-04 ` .
4274
43- Let's find the first nightly where this fails. First you need to determine a
44- nightly release where the code works, and one where it doesn't. We can see
45- from the issue that we know stable 1.28.0 was working, and that it no longer
46- works as of nightly 2018-08-04. We're testing against nightlies, and stable
47- 1.28.0 branched off master 12 weeks prior to its release, and the regression
48- could have been introduced to master during that time, so I'll just take a
49- guess that nightly 2018-05-07 is OK. Don't worry if you guess wrong,
50- ` cargo-bisect-rustc ` will tell you and you can just pick a larger date range.
51- To find the exact nightly release where it stopped working:
75+ We need to provide the end point for this particular example because
76+ that's an old regression already fixed on the latests nightlies.
77+ We could also provide a start point if we know one, that's going to make
78+ the tool avoid searching for that so answer our request faster.
79+ For instance:
5280
5381```
54- cargo- bisect-rustc --test-dir=foo --start=2018-05-07 --end=2018-08-04
82+ cargo bisect-rustc --test-dir=foo --start=2018-05-07 --end=2018-08-04
5583```
5684
5785By default it will run ` cargo build ` in the project and check whether or not
@@ -62,11 +90,15 @@ it fails. In just a few steps, we find that it stopped working on
6290> artifacts for future runs. They are stored in the normal location for your
6391> toolchains in ` RUSTUP_HOME ` .
6492
65- ## Regressing to a PR
93+ After that is going to automatically search for the commit that
94+ introduced the regression.
95+
96+ ## Finding a regression between commits
6697
67- But wait, we can do better! As long as the regression wasn't too long ago, we
68- can find the exact PR that caused the regression. Use git hashes from the
69- rustc repo's log as the start/end parameters. They must be from bors on the
98+ We can also just ask the tool to look between commits if that's what we
99+ want. As long as the regression wasn't too long ago, we can find the
100+ exact PR that caused the regression. Use git hashes from the rustc
101+ repo's log as the start/end parameters. They must be from bors on the
70102master branch.
71103
72104To find a list of all such usable commit hashes, we can use ` git log ` in the
0 commit comments