1
1
# Bisecting Regressions
2
2
3
3
The [ ` cargo-bisect-rustc ` ] tool makes it super easy to find exactly when
4
- behavior has changed in rustc. It automatically downloads rustc artifacts and
5
- tests them against a project you provide until it finds the regression. To
6
- install:
4
+ behavior has regressed in rustc. It automatically downloads rustc
5
+ artifacts and tests them against a project you provide until it finds
6
+ the regression.
7
+
8
+ To install the tool run:
7
9
8
10
``` sh
9
- git clone https://github.com/rust-lang-nursery/cargo-bisect-rustc.git
10
- RUST_SRC_REPO=/path/to/rust cargo install --path=cargo-bisect-rustc
11
+ cargo install cargo-bisect-rustc
12
+ ```
13
+
14
+ or (to avoid cloning rustc)
15
+
16
+ ```
17
+ RUST_SRC_REPO=/path/to/rust cargo install cargo-bisect-rustc
11
18
```
12
19
13
20
The ` RUST_SRC_REPO ` should be a path to a git clone of the rust repo. If you
14
21
don't specify it, it will look in the current directory for ` rust.git ` or
15
22
check it out automatically if it's not there (only necessary if doing git hash
16
23
bisections).
17
24
25
+ First, if you have a nightly version of the compiler already installed
26
+ as the default toolchain and you don't pass an end flag, the tool is
27
+ going to assume that that's the version that has regressed and use it as
28
+ the "bad" version. Otherwise would use that start point or just use the
29
+ latest nightly.
30
+ If you have provided a start flag it would use that as the "good"
31
+ version, otherwise is going to search for a good one backwards.
32
+
33
+ Once the tool has an start point (good version) and end point (bad
34
+ version), is going to do the bisect to find the regressed nightly. Once
35
+ it finds the regressed nightly is going to look between that nightly and
36
+ the one of the day before for the specific commit that has the
37
+ regression and report everything back to the user.
38
+
39
+ So, there's a bunch of ways to run the tool, the easiest one is to
40
+ allow the tool to do the job and just run ` cargo bisect-rustc ` on the
41
+ project and let the tool figure things out.
42
+
43
+ Note: you can also pass ` RUST_SRC_REPO ` env var, should be a path to a
44
+ git clone of the rust repo. If you don't specify it, it will look in the
45
+ current directory for ` rust.git ` or check it out automatically if it's
46
+ not there (only necessary if doing git hash bisections).
47
+
48
+ ## Finding a regression
49
+
18
50
Create a cargo project that demonstrates the regression. Let's use
19
51
[ issue #53157 ] as an example:
20
52
@@ -38,20 +70,16 @@ fn main() {
38
70
}
39
71
```
40
72
41
- ## Regressing to a nightly
73
+ Then run ` cargo bisect-rustc --end=2018-08-04 ` .
42
74
43
- Let's find the first nightly where this fails. First you need to determine a
44
- nightly release where the code works, and one where it doesn't. We can see
45
- from the issue that we know stable 1.28.0 was working, and that it no longer
46
- works as of nightly 2018-08-04. We're testing against nightlies, and stable
47
- 1.28.0 branched off master 12 weeks prior to its release, and the regression
48
- could have been introduced to master during that time, so I'll just take a
49
- guess that nightly 2018-05-07 is OK. Don't worry if you guess wrong,
50
- ` cargo-bisect-rustc ` will tell you and you can just pick a larger date range.
51
- To find the exact nightly release where it stopped working:
75
+ We need to provide the end point for this particular example because
76
+ that's an old regression already fixed on the latests nightlies.
77
+ We could also provide a start point if we know one, that's going to make
78
+ the tool avoid searching for that so answer our request faster.
79
+ For instance:
52
80
53
81
```
54
- cargo- bisect-rustc --test-dir=foo --start=2018-05-07 --end=2018-08-04
82
+ cargo bisect-rustc --test-dir=foo --start=2018-05-07 --end=2018-08-04
55
83
```
56
84
57
85
By default it will run ` cargo build ` in the project and check whether or not
@@ -62,11 +90,15 @@ it fails. In just a few steps, we find that it stopped working on
62
90
> artifacts for future runs. They are stored in the normal location for your
63
91
> toolchains in ` RUSTUP_HOME ` .
64
92
65
- ## Regressing to a PR
93
+ After that is going to automatically search for the commit that
94
+ introduced the regression.
95
+
96
+ ## Finding a regression between commits
66
97
67
- But wait, we can do better! As long as the regression wasn't too long ago, we
68
- can find the exact PR that caused the regression. Use git hashes from the
69
- rustc repo's log as the start/end parameters. They must be from bors on the
98
+ We can also just ask the tool to look between commits if that's what we
99
+ want. As long as the regression wasn't too long ago, we can find the
100
+ exact PR that caused the regression. Use git hashes from the rustc
101
+ repo's log as the start/end parameters. They must be from bors on the
70
102
master branch.
71
103
72
104
To find a list of all such usable commit hashes, we can use ` git log ` in the
0 commit comments