You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: text/3127-trim-paths.md
+88-44Lines changed: 88 additions & 44 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,8 +7,15 @@
7
7
[summary]: #summary
8
8
9
9
Cargo should have a [profile setting](https://doc.rust-lang.org/cargo/reference/profiles.html#profile-settings) named `trim-paths`
10
-
to sanitise absolute paths introduced during compilation that may be embedded in the compilation output. This should be enabled by default for
11
-
`release` profile.
10
+
to sanitise absolute paths introduced during compilation that may be embedded in the compiled binary executable or library, and optionally in
11
+
the separate debug symbols file (depending on `split-debuginfo` settings).
12
+
13
+
`cargo build` with the default `release` profile should not produce any host filesystem dependent paths into binary executable or library. But
14
+
it will retain the paths in separate debug symbols file, if one exists, to help debuggers and profilers locate the source files.
15
+
16
+
To facilitate this, a new flag named `--remap-scope` should be added to `rustc` controlling the behaviour of `--remap-path-prefix`, allowing us to fine
17
+
tune the scope of remapping, speicifying paths under which context (in marco expansion, in debuginfo or in diagnostics)
18
+
should or shouldn't be remapped.
12
19
13
20
# Motivation
14
21
[motivation]: #motivation
@@ -66,86 +73,119 @@ This is undesirable for the following reasons:
66
73
## Handling sysroot paths
67
74
At the moment, paths to the source files of standard and core libraries, even when they are present, always begin with a virtual prefix in the form
68
75
of `/rustc/[SHA1 hash]/library`. This is not an issue when the source files are not present (i.e. when `rust-src` component is not installed), but
69
-
when a user installs `rust-src` they expect the path to their local copy of source files to be visible. Hence the user should be given an option for
70
-
the local paths to show up in panic messages and backtraces.
76
+
when a user installs `rust-src` they may want the path to their local copy of source files to be visible. Hence the default behaviour when `rust-src`
77
+
is installed should be to embed the local path. These local paths should be then affected by path remappings in the usual way.
78
+
79
+
## Preserving debuginfo to help debuggers
80
+
At the moment, `--remap-path-prefix` will cause paths to source files in debuginfo to be remapped. On platforms where the debuginfo resides in a
81
+
separate file from the distributable binary, this may be unnecessary and it prevents debuggers from being able to find the source. Hence `rustc`
82
+
should support finer grained control over paths in which contexts should be remapped.
`trim-paths` is a profile setting which can be set to either `true` or `false`. This is enabled by default when you do a release build,
76
-
such as via `cargo build --release`. You can also manually override it by specifying this option in `Cargo.toml`:
87
+
## The rustc book: Command-line arguments
88
+
89
+
### `--remap-scope`: configure the scope of path remapping
90
+
91
+
When the `--remap-path-prefix` option is passed to rustc then source path prefixes in all output will be affected.
92
+
The `--remap-scope` argument can be used in conjunction with `--remap-path-prefix` to determine paths in which output context should be affected.
93
+
This flag accepts a comma-separated list of values and may be specified multiple times. The valid scopes are:
94
+
95
+
-`macro` - apply remappings to the expansion of `std::file!()` macro. This is where paths in embedded panic messages come from
96
+
-`debuginfo` - apply remappings to debug information
97
+
-`diagnostics` - apply remappings to printed compiler diagnostics
98
+
99
+
## Cargo
100
+
101
+
`trim-paths` is a profile setting which controls the sanitisation of file paths in compilation outputs. It has three valid options:
102
+
-`0` or `false`: no sanitisation at all
103
+
-`1`: sanitise only the paths in emitted executable or library binaries. It always affects paths from macros such as panic messages, and in debug information
104
+
only if they will be embedded together with the binary (the default on platforms with ELF binaries, such as Linux and windows-gnu),
105
+
but will not touch them if they are in a separate symbols file (the default on Windows MSVC and macOS)
106
+
-`2` or `ture`: sanitise paths in all compilation outputs, including compiled executable/library, separate symbols file (if one exists), and compiler diagnostics.
107
+
108
+
The default release profile uses option `1`. You can also manually override it by specifying this option in `Cargo.toml`:
77
109
```toml
78
110
[profile.dev]
79
-
trim-paths = true
111
+
trim-paths = 2
80
112
81
113
[profile.release]
82
-
trim-paths = false
114
+
trim-paths = 0
83
115
```
84
116
85
-
With `trim-paths` option enabled, the compilation process will not introduce any absolute paths into the build output. Instead, paths containing
86
-
certain prefixes will be replaced with something stable by the following rules:
117
+
When a path is in scope for sanitisation, it is replaced with the following rules:
87
118
88
-
1. Path to the source files of the standard and core library will begin with `/rustc/[rustc version]`.
119
+
1. Path to the source files of the standard and core library (sysroot) will begin with `/rustc/[rustc commit hash]`.
89
120
E.g. `/home/username/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs` ->
2. Path to the working directory will be replaced with `.`. E.g. `/home/username/crate/src/lib.rs` -> `./src/lib.rs`.
92
123
3. Path to packages outside of the working directory will be replaced with `[package name]-[version]`. E.g. `/home/username/deps/foo/src/lib.rs` -> `foo-0.1.0/src/lib.rs`
93
124
94
-
If using MSVC toolchain, path to the .pdb file containing debug information are be embedded as the file name of the .pdb file only, wihtout any path
95
-
information.
96
-
97
-
With `trim-paths` option disabled, the embedding of path to the source files of the standard and core library will depend on if `rust-src` component is present. If it is, then the real path pointing to a copy of the source files on your file system will be embedded; if it isn't, then they will
98
-
show up as `/rustc/[rustc version]/library/...` (just like when `trim-paths` is enabled). Paths to all other source files will not be affected.
125
+
When a path to the source files of the standard and core library is *not* in scope for sanitisation, the emitted path will depend on if `rust-src` component
126
+
is present. If it is, then the real path pointing to a copy of the source files on your file system will be emitted; if it isn't, then they will
127
+
show up as `/rustc/[rustc commit hash]/library/...` (just like when it is selected for sanitisation). Paths to all other source files will not be affected.
99
128
100
-
Note that this will not affect any hard-coded paths in the source code.
129
+
This will not affect any hard-coded paths in the source code.
We only need to change the behaviour for `Test` and `Build` compile modes.
107
137
108
-
If `trim-paths` is enabled, Cargo will emit two `--remap-path-prefix` arguments to `rustc` for each compilation unit. One mapping is from the path of
109
-
the local sysroot to `/rustc/[rust version]`. The other mapping depends on if the package containing the compilation unit is under the working
110
-
directory. If it is, then the mapping is from the absolute path to the working directory to `.`. If it's outside the working directory, then the
111
-
mapping is from the absolute path of the package root to `[package name]-[package version]`.
138
+
If `trim-paths` is `0` (`false`), no extra flag is supplied to `rustc`.
139
+
140
+
If `trip-paths` is `1` or `2` (`true`), then two `--remap-path-prefix` arguments are supplied to `rustc`:
141
+
- From the path of the local sysroot to `/rustc/[commit hash]`.
142
+
- If the compilation unit is under the working directory, from the absolute path to the working directory to `.`.
143
+
If it's outside the working directory, from the absolute path of the package root to `[package name]-[package version]`.
144
+
145
+
A further `--remap-scope` is also supplied for options `1` and `2`:
146
+
147
+
If `trim-path` is `1`, then it depends on the setting of `split-debuginfo` (whether the setting is explicitly supplied or from the default)
148
+
- If `split-debuginfo` is `off`, then `--remap-scope=macro,debuginfo`.
149
+
- If `split-debuginfo` is `packed` or `unpacked`, then `--remap-scope=macro`
150
+
This is because we always want to remap panic messages as they will always be embedded in executable/library, but we don't need to touch the separate
151
+
symbols file
152
+
153
+
If `trim-path` is `2` (`true`), all paths will be affected, equivalent to `--remap-scope=macro,debuginfo,diagnostics`
112
154
113
-
Some interactions with compiler-intrinstic macros need to be considered, though these are entirely down to `rustc`'s implementation of
114
-
`--remap-path-prefix`:
155
+
156
+
Some interactions with compiler-intrinstic macros need to be considered:
115
157
1. Path (of the current file) introduced by [`file!()`](https://doc.rust-lang.org/std/macro.file.html)*will* be remapped. **Things may break** if
116
158
the code interacts with its own source file at runtime by using this macro.
117
159
2. Path introduced by [`include!()`](https://doc.rust-lang.org/std/macro.include.html)*will* be remapped, given that the included file is under
118
160
the current working directory or a dependency package.
119
161
120
-
If the user further supplies custom `--remap-path-prefix` arguments via `RUSTFLAGS` or similar mechanisms, they will take precedence over the one
121
-
supplied by `trim-paths`. This means that the user-defined `--remap-path-prefix`s must be supplied *after* Cargo's own remapping.
162
+
If the user further supplies custom `--remap-path-prefix` arguments via `RUSTFLAGS`
163
+
or similar mechanisms, they will take precedence over the one supplied by `trim-paths`. This means that the user-defined remapping arguments must be
164
+
supplied *after* Cargo's own remapping.
165
+
122
166
123
167
Additionally, when using MSVC linker, Cargo should emit `/PDBALTPATH:%_PDB%` to the linker via `-C link-arg`. This makes the linker embed
124
168
only the file name of the .pdb file without the path to it.
125
169
126
-
## Changing handling of sysroot path
127
-
The virtualisation of sysroot files to `/rustc/[SHA1 hash]/library/...` was done at compiler bootstraping, specifically when
170
+
## Changing handling of sysroot path in `rustc`
171
+
172
+
The virtualisation of sysroot files to `/rustc/[commit hash]/library/...` was done at compiler bootstraping, specifically when
128
173
`remap-debuginfo = true` in `config.toml`. This is done for Rust distribution on all channels.
129
174
130
-
At `rustc` runtime (i.e. compiling some code), we try to correlate this virtual path to a real path pointing to the file on the local file system.
131
-
Currently the result is represented internally as if the path was remapped by `--remap-path-prefix`, holding both the virtual name and local path.
175
+
At `rustc` runtime (i.e. compiling some code), we try to correlate this virtual path to a real path pointing to the file on the local file system
176
+
Currently the result is represented internally as if the path was remapped by a `--remap-path-prefix`, from local `rust-src` path to the virtual path.
132
177
Only the virtual name is ever emitted for metadata or codegen. We want to change this behaviour such that, when `rust-src` source files can be
133
-
discovered, the virtual path is discarded and therefore will be embedded unless being remapped by`--remap-path-prefix`in the usual way. The relevant part of the code is here:
discovered, the virtual path is discarded and therefore the local path will be embedded, unless there is a`--remap-path-prefix`that causes this
179
+
local path to be remapped in the usual way.
135
180
136
-
We would also like to change the virtualisation of sysroot to `/rustc/[rustc version]/library/...`, instead of the rustc commit hash. This is shorter and more helpful as an identifier, and makes `trim-paths` easier to implement: to make the embedded path the same whether or not `rust-src` is installed, we need to emit the same sysroot virutalisation as was done during bootstrapping. Getting the version number is easier than getting the commit hash. The relevant part of the code is here: https://github.com/rust-lang/rust/blob/d8af907491e20339e41d048d6a32b41ddfa91dfe/src/bootstrap/lib.rs#L831-L834
137
181
138
182
# Drawbacks
139
183
[drawbacks]: #drawbacks
140
184
141
-
With `trim-paths` enabled, if the `debug` option is simultaneously not `false` (it is turned off by default under `release` profile), paths in
142
-
debuginfo will also be remapped. Debuggers will no longer be able to automatically discover and load source files outside of the working directory.
143
-
This can be remidated by [debugger features](https://lldb.llvm.org/use/map.html#miscellaneous) remapping the path back to a filesystem path.
144
-
145
-
The user also will not be able to `Ctrl+click` on any paths provided in panic messages or backtraces outside of the working directory. But
185
+
The user will not be able to `Ctrl+click` on any paths provided in panic messages or backtraces outside of the working directory. But
146
186
there shouldn't be any confusion as the combination of pacakge name and version can be used to pinpoint the file.
147
187
148
-
As mentioned above, `trim-paths` may break code that relies on `file!()` to evaluate to an accessible path to the file. Hence enabling
188
+
As mentioned above, `trim-paths` may break code that relies on `std::file!()` to evaluate to an accessible path to the file. Hence enabling
149
189
it by default for release builds may be a technically breaking change. Occurances of such use should be extremely rare but should be investigated
150
190
via a Crater run. In case this breakage is unacceptable, `trim-paths` can be made an opt-in option rather than default in any build profile.
151
191
@@ -160,10 +200,10 @@ Path to sysroot crates are specially handled by `rustc`. Due to this, the behavi
160
200
Although good for privacy and reproducibility, some people find it a hinderance for debugging: https://github.com/rust-lang/rust/issues/85463.
161
201
Hence the user should be given control on if they want the virtual or local path.
162
202
163
-
One alternative for the sysroot handling is to keep the logic in `rustc` largely the same, always emitting the virutalised path by default, and
164
-
then introduce an extra option named `--embed-local-sysroot` to embed the local paths if the source files can be found. This inovles adding an extra
165
-
option to `rustc` and prevents any uniformity in `--remap-path-prefix`'s handling over sysroot paths, compared to other paths (it currently doesn't
166
-
affect sysroot paths at all).
203
+
An alternative to `--remap-scope`is to have individual `--remap-path-prefxi`-like flags, one each for macro, debuginfo and diagnostics, requiring
204
+
the full mapping to be given for each context. This is similar to what GCC and Clang does as described below, but we have added a third context
205
+
for diagnostics. This technically enables for even finer grained control, allowing different paths in different
206
+
contexts to be remapped differently. However it will cause the command line to be very verbose under most normal use cases.
167
207
168
208
# Prior art
169
209
[prior-art]: #prior-art
@@ -174,13 +214,17 @@ The name `trim-paths` came from the [similar feature](https://golang.org/cmd/go/
174
214
Go does not enable this by default. Since Go does not differ between debug and release builds, removing absolute paths for all build would be
175
215
a hassle for debugging. However this is not an issue for Rust as we have separate debug build profile.
176
216
217
+
GCC and Clang both have a flag equivalent to `--remap-path-prefix`, but they also both have two separate flags one for only macro expansion and
218
+
the other for only debuginfo: https://reproducible-builds.org/docs/build-path/. This is the origin of the `--remap-scope` idea.
219
+
177
220
# Unresolved questions
178
221
[unresolved-questions]: #unresolved-questions
179
222
180
223
- Should we treat the current working directory the same as other packages? We could have one fewer remapping rule by remapping all
181
224
package roots to `[package name]-[version]`. A minor downside to this is not being able to `Ctrl+click` on paths to files the user is working
182
225
on from panic messages.
183
-
- Should we use a slightly more complex remapping rule, like distinguishing packages from registry, git and path, as mentioned in https://github.com/rust-lang/rust/issues/40552?
226
+
- Should we use a slightly more complex remapping rule, like distinguishing packages from registry, git and path, as mentioned in
227
+
https://github.com/rust-lang/rust/issues/40552?
184
228
- Will these cover all potentially embedded paths? Have we missed anything?
185
229
- Should we make this affect more `CompileMode`s, such as `Check`, where the emitted `rmeta` file will also contain absolute paths?
0 commit comments