diff --git a/rust-version b/rust-version index 6ec700b9b..a399b5cd7 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -6bcdcc73bd11568fd85f5a38b58e1eda054ad1cd +425a9c0a0e365c0b8c6cfd00c2ded83a73bed9a0 diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 9ded467d5..025a078ae 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -107,7 +107,6 @@ - [Installation](./autodiff/installation.md) - [How to debug](./autodiff/debugging.md) - [Autodiff flags](./autodiff/flags.md) - - [Current limitations](./autodiff/limitations.md) # Source Code Representation diff --git a/src/autodiff/limitations.md b/src/autodiff/limitations.md deleted file mode 100644 index 90afbd51f..000000000 --- a/src/autodiff/limitations.md +++ /dev/null @@ -1,27 +0,0 @@ -# Current limitations - -## Safety and Soundness - -Enzyme currently assumes that the user passes shadow arguments (`dx`, `dy`, ...) of appropriate size. Under Reverse Mode, we additionally assume that shadow arguments are mutable. In Reverse Mode we adjust the outermost pointer or reference to be mutable. Therefore `&f32` will receive the shadow type `&mut f32`. However, we do not check length for other types than slices (e.g. enums, Vec). We also do not enforce mutability of inner references, but will warn if we recognize them. We do intend to add additional checks over time. - -## ABI adjustments - -In some cases, a function parameter might get lowered in a way that we currently don't handle correctly, leading to a compile time type mismatch in the `rustc_codegen_llvm` backend. Here are some [examples](https://github.com/EnzymeAD/rust/issues/105). - -## Compile Times - -Enzyme will often achieve excellent runtime performance, but might increase your compile time by a large factor. For Rust, we already have made significant improvements and have a list of further improvements planed - please reach out if you have time to help here. - -### Type Analysis - -Most of the times, Type Analysis (TA) is the reason of large (>5x) compile time increases when using Enzyme. This poster explains why we need to run Type Analysis in the bottom left part: [Poster Link](https://c.wsmoses.com/posters/Enzyme-llvmdev.pdf). - -We intend to increase the number of locations where we pass down Type information based on Rust types, which in turn will reduce the number of locations where Enzyme has to run Type Analysis, which will help compile times. - -### Duplicated Optimizations - -The key reason for Enzyme offering often excellent performance is that Enzyme differentiates already optimized LLVM-IR. However, we also (have to) run LLVM's optimization pipeline after differentiating, to make sure that the code which Enzyme generates is optimized properly. As a result you should have excellent runtime performance (please fill an issue if not), but at a compile time cost for running optimizations twice. - -### Fat-LTO - -The usage of `#[autodiff(...)]` currently requires compiling your project with Fat-LTO. We technically only need LTO if the function being differentiated calls functions in other compilation units. Therefore, other solutions are possible, but this is the most simple one to get started. diff --git a/src/building/bootstrapping/debugging-bootstrap.md b/src/building/bootstrapping/debugging-bootstrap.md index 9c5ebbd36..fb90c0fdb 100644 --- a/src/building/bootstrapping/debugging-bootstrap.md +++ b/src/building/bootstrapping/debugging-bootstrap.md @@ -1,120 +1,100 @@ # Debugging bootstrap -There are two main ways to debug bootstrap itself. The first is through println logging, and the second is through the `tracing` feature. - -> FIXME: this section should be expanded +There are two main ways of debugging (and profiling bootstrap). The first is through println logging, and the second is through the `tracing` feature. ## `println` logging Bootstrap has extensive unstructured logging. Most of it is gated behind the `--verbose` flag (pass `-vv` for even more detail). -If you want to know which `Step` ran a command, you could invoke bootstrap like so: +If you want to see verbose output of executed Cargo commands and other kinds of detailed logs, pass `-v` or `-vv` when invoking bootstrap. Note that the logs are unstructured and may be overwhelming. ``` $ ./x dist rustc --dry-run -vv learning about cargo running: RUSTC_BOOTSTRAP="1" "/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "metadata" "--format-version" "1" "--no-deps" "--manifest-path" "/home/jyn/src/rust2/Cargo.toml" (failure_mode=Exit) (created at src/bootstrap/src/core/metadata.rs:81:25, executed at src/bootstrap/src/core/metadata.rs:92:50) running: RUSTC_BOOTSTRAP="1" "/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "metadata" "--format-version" "1" "--no-deps" "--manifest-path" "/home/jyn/src/rust2/library/Cargo.toml" (failure_mode=Exit) (created at src/bootstrap/src/core/metadata.rs:81:25, executed at src/bootstrap/src/core/metadata.rs:92:50) -> Assemble { target_compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu } } - > Libdir { compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu }, target: x86_64-unknown-linux-gnu } - > Sysroot { compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu }, force_recompile: false } -Removing sysroot /home/jyn/src/rust2/build/tmp-dry-run/x86_64-unknown-linux-gnu/stage1 to avoid caching bugs - < Sysroot { compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu }, force_recompile: false } - < Libdir { compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu }, target: x86_64-unknown-linux-gnu } -... -``` - -This will go through all the recursive dependency calculations, where `Step`s internally call `builder.ensure()`, without actually running cargo or the compiler. - -In some cases, even this may not be enough logging (if so, please add more!). In that case, you can omit `--dry-run`, which will show the normal output inline with the debug logging: - -``` - c Sysroot { compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu }, force_recompile: false } -using sysroot /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage0-sysroot -Building stage0 library artifacts (x86_64-unknown-linux-gnu) -running: cd "/home/jyn/src/rust2" && env ... RUSTC_VERBOSE="2" RUSTC_WRAPPER="/home/jyn/src/rust2/build/bootstrap/debug/rustc" "/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-Zroot-dir=/home/jyn/src/rust2" "-v" "-v" "--manifest-path" "/home/jyn/src/rust2/library/sysroot/Cargo.toml" "--message-format" "json-render-diagnostics" - 0.293440230s INFO prepare_target{force=false package_id=sysroot v0.0.0 (/home/jyn/src/rust2/library/sysroot) target="sysroot"}: cargo::core::compiler::fingerprint: fingerprint error for sysroot v0.0.0 (/home/jyn/src/rust2/library/sysroot)/Build/TargetInner { name_inferred: true, ..: lib_target("sysroot", ["lib"], "/home/jyn/src/rust2/library/sysroot/src/lib.rs", Edition2021) } ... ``` -In most cases this should not be necessary. +## `tracing` in bootstrap -TODO: we should convert all this to structured logging so it's easier to control precisely. +Bootstrap has a conditional `tracing` feature, which provides the following features: +- It enables structured logging using [`tracing`][tracing] events and spans. +- It generates a [Chrome trace file] that can be used to visualize the hierarchy and durations of executed steps and commands. + - You can open the generated `chrome-trace.json` file using Chrome, on the `chrome://tracing` tab, or e.g. using [Perfetto]. +- It generates [GraphViz] graphs that visualize the dependencies between executed steps. + - You can open the generated `step-graph-*.dot` file using e.g. [xdot] to visualize the step graph, or use e.g. `dot -Tsvg` to convert the GraphViz file to an SVG file. +- It generates a command execution summary, which shows which commands were executed, how many of their executions were cached, and what commands were the slowest to run. + - The generated `command-stats.txt` file is in a simple human-readable format. -## `tracing` in bootstrap +The structured logs will be written to standard error output (`stderr`), while the other outputs will be stored in files in the `/bootstrap-trace/` directory. For convenience, bootstrap will also create a symlink to the latest generated trace output directory at `/bootstrap-trace/latest`. -Bootstrap has conditional [`tracing`][tracing] setup to provide structured logging. +> Note that if you execute bootstrap with `--dry-run`, the tracing output directory might change. Bootstrap will always print a path where the tracing output files were stored at the end of its execution. [tracing]: https://docs.rs/tracing/0.1.41/tracing/index.html +[Chrome trace file]: https://www.chromium.org/developers/how-tos/trace-event-profiling-tool/ +[Perfetto]: https://ui.perfetto.dev/ +[GraphViz]: https://graphviz.org/doc/info/lang.html +[xdot]: https://github.com/jrfonseca/xdot.py ### Enabling `tracing` output -Bootstrap will conditionally build `tracing` support and enable `tracing` output if the `BOOTSTRAP_TRACING` env var is set. - -#### Basic usage - -Example basic usage[^just-trace]: +To enable the conditional `tracing` feature, run bootstrap with the `BOOTSTRAP_TRACING` environment variable. -[^just-trace]: It is not recommended to use *just* `BOOTSTRAP_TRACING=TRACE` because that will dump *everything* at `TRACE` level, including logs intentionally gated behind custom targets as they are too verbose even for `TRACE` level by default. +[tracing_subscriber filter]: https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html ```bash -$ BOOTSTRAP_TRACING=bootstrap=TRACE ./x build library --stage 1 +$ BOOTSTRAP_TRACING=trace ./x build library --stage 1 ``` Example output[^unstable]: ``` -$ BOOTSTRAP_TRACING=bootstrap=TRACE ./x check src/bootstrap/ +$ BOOTSTRAP_TRACING=trace ./x build library --stage 1 --dry-run Building bootstrap - Compiling bootstrap v0.0.0 (/home/joe/repos/rust/src/bootstrap) - Finished `dev` profile [unoptimized] target(s) in 2.74s - DEBUG bootstrap parsing flags - bootstrap::core::config::flags::Flags::parse args=["check", "src/bootstrap/"] - DEBUG bootstrap parsing config based on flags - DEBUG bootstrap creating new build based on config - bootstrap::Build::build - TRACE bootstrap setting up job management - TRACE bootstrap downloading rustfmt early - bootstrap::handling hardcoded subcommands (Format, Suggest, Perf) - DEBUG bootstrap not a hardcoded subcommand; returning to normal handling, cmd=Check { all_targets: false } - DEBUG bootstrap handling subcommand normally - bootstrap::executing real run - bootstrap::(1) executing dry-run sanity-check - bootstrap::(2) executing actual run -Checking stage0 library artifacts (x86_64-unknown-linux-gnu) - Finished `release` profile [optimized + debuginfo] target(s) in 0.04s -Checking stage0 compiler artifacts {rustc-main, rustc_abi, rustc_arena, rustc_ast, rustc_ast_ir, rustc_ast_lowering, rustc_ast_passes, rustc_ast_pretty, rustc_attr_data_structures, rustc_attr_parsing, rustc_baked_icu_data, rustc_borrowck, rustc_builtin_macros, rustc_codegen_llvm, rustc_codegen_ssa, rustc_const_eval, rustc_data_structures, rustc_driver, rustc_driver_impl, rustc_error_codes, rustc_error_messages, rustc_errors, rustc_expand, rustc_feature, rustc_fluent_macro, rustc_fs_util, rustc_graphviz, rustc_hir, rustc_hir_analysis, rustc_hir_pretty, rustc_hir_typeck, rustc_incremental, rustc_index, rustc_index_macros, rustc_infer, rustc_interface, rustc_lexer, rustc_lint, rustc_lint_defs, rustc_llvm, rustc_log, rustc_macros, rustc_metadata, rustc_middle, rustc_mir_build, rustc_mir_dataflow, rustc_mir_transform, rustc_monomorphize, rustc_next_trait_solver, rustc_parse, rustc_parse_format, rustc_passes, rustc_pattern_analysis, rustc_privacy, rustc_query_impl, rustc_query_system, rustc_resolve, rustc_sanitizers, rustc_serialize, rustc_session, rustc_smir, rustc_span, rustc_symbol_mangling, rustc_target, rustc_trait_selection, rustc_traits, rustc_transmute, rustc_ty_utils, rustc_type_ir, rustc_type_ir_macros, stable_mir} (x86_64-unknown-linux-gnu) - Finished `release` profile [optimized + debuginfo] target(s) in 0.23s -Checking stage0 bootstrap artifacts (x86_64-unknown-linux-gnu) - Checking bootstrap v0.0.0 (/home/joe/repos/rust/src/bootstrap) - Finished `release` profile [optimized + debuginfo] target(s) in 0.64s - DEBUG bootstrap checking for postponed test failures from `test --no-fail-fast` -Build completed successfully in 0:00:08 + Finished `dev` profile [unoptimized] target(s) in 0.05s +15:56:52.477 INFO > tool::LibcxxVersionTool {target: x86_64-unknown-linux-gnu} (builder/mod.rs:1715) +15:56:52.575 INFO > compile::Assemble {target_compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }} (builder/mod.rs:1715) +15:56:52.575 INFO > tool::Compiletest {compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target: x86_64-unknown-linux-gnu} (builder/mod.rs:1715) +15:56:52.576 INFO > tool::ToolBuild {build_compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target: x86_64-unknown-linux-gnu, tool: "compiletest", path: "src/tools/compiletest", mode: ToolBootstrap, source_type: InTree, extra_features: [], allow_features: "internal_output_capture", cargo_args: [], artifact_kind: Binary} (builder/mod.rs:1715) +15:56:52.576 INFO > builder::Libdir {compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target: x86_64-unknown-linux-gnu} (builder/mod.rs:1715) +15:56:52.576 INFO > compile::Sysroot {compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, force_recompile: false} (builder/mod.rs:1715) +15:56:52.578 INFO > compile::Assemble {target_compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }} (builder/mod.rs:1715) +15:56:52.578 INFO > tool::Compiletest {compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target: x86_64-unknown-linux-gnu} (builder/mod.rs:1715) +15:56:52.578 INFO > tool::ToolBuild {build_compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target: x86_64-unknown-linux-gnu, tool: "compiletest", path: "src/tools/compiletest", mode: ToolBootstrap, source_type: InTree, extra_features: [], allow_features: "internal_output_capture", cargo_args: [], artifact_kind: Binary} (builder/mod.rs:1715) +15:56:52.578 INFO > builder::Libdir {compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target: x86_64-unknown-linux-gnu} (builder/mod.rs:1715) +15:56:52.578 INFO > compile::Sysroot {compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, force_recompile: false} (builder/mod.rs:1715) + Finished `release` profile [optimized] target(s) in 0.11s +Tracing/profiling output has been written to /build/bootstrap-trace/latest +Build completed successfully in 0:00:00 ``` +[^unstable]: This output is always subject to further changes. + #### Controlling tracing output -The env var `BOOTSTRAP_TRACING` accepts a [`tracing` env-filter][tracing-env-filter]. +The environment variable `BOOTSTRAP_TRACING` accepts a [`tracing_subscriber` filter][tracing-env-filter]. If you set `BOOTSTRAP_TRACING=trace`, you will enable all logs, but that can be overwhelming. You can thus use the filter to reduce the amount of data logged. There are two orthogonal ways to control which kind of tracing logs you want: -1. You can specify the log **level**, e.g. `DEBUG` or `TRACE`. -2. You can also control the log **target**, e.g. `bootstrap` or `bootstrap::core::config` vs custom targets like `CONFIG_HANDLING`. - - Custom targets are used to limit what is output when `BOOTSTRAP_TRACING=bootstrap=TRACE` is used, as they can be too verbose even for `TRACE` level by default. Currently used custom targets: - - `CONFIG_HANDLING` - -The `TRACE` filter will enable *all* `trace` level or less verbose level tracing output. +1. You can specify the log **level**, e.g. `debug` or `trace`. + - If you select a level, all events/spans with an equal or higher priority level will be shown. +2. You can also control the log **target**, e.g. `bootstrap` or `bootstrap::core::config` or a custom target like `CONFIG_HANDLING` or `STEP`. + - Custom targets are used to limit what kinds of spans you are interested in, as the `BOOTSTRAP_TRACING=trace` output can be quite verbose. Currently, you can use the following custom targets: + - `CONFIG_HANDLING`: show spans related to config handling + - `STEP`: show all executed steps. Note that executed commands have `info` event level. + - `COMMAND`: show all executed commands. Note that executed commands have `trace` event level. You can of course combine them (custom target logs are typically gated behind `TRACE` log level additionally): ```bash -$ BOOTSTRAP_TRACING=CONFIG_HANDLING=TRACE ./x build library --stage 1 +$ BOOTSTRAP_TRACING=CONFIG_HANDLING=trace,STEP=info,COMMAND=trace ./x build library --stage 1 ``` -[^unstable]: This output is always subject to further changes. - [tracing-env-filter]: https://docs.rs/tracing-subscriber/0.3.19/tracing_subscriber/filter/struct.EnvFilter.html +Note that the level that you specify using `BOOTSTRAP_TRACING` also has an effect on the spans that will be recorded in the Chrome trace file. + ##### FIXME(#96176): specific tracing for `compiler()` vs `compiler_for()` The additional targets `COMPILER` and `COMPILER_FOR` are used to help trace what @@ -123,12 +103,6 @@ if [#96176][cleanup-compiler-for] is resolved. [cleanup-compiler-for]: https://github.com/rust-lang/rust/issues/96176 -### Rendering step graph - -When you run bootstrap with the `BOOTSTRAP_TRACING` environment variable configured, bootstrap will automatically output a DOT file that shows all executed steps and their dependencies. The files will have a prefix `bootstrap-steps`. You can use e.g. `xdot` to visualize the file or e.g. `dot -Tsvg` to convert the DOT file to a SVG file. - -A separate DOT file will be outputted for dry-run and non-dry-run execution. - ### Using `tracing` in bootstrap Both `tracing::*` macros and the `tracing::instrument` proc-macro attribute need to be gated behind `tracing` feature. Examples: @@ -149,15 +123,6 @@ impl Step for Foo { todo!() } - #[cfg_attr( - feature = "tracing", - instrument( - level = "trace", - name = "Foo::run", - skip_all, - fields(compiler = ?builder.compiler), - ), - )] fn run(self, builder: &Builder<'_>) -> Self::Output { trace!(?run, "entered Foo::run"); @@ -172,21 +137,6 @@ For `#[instrument]`, it's recommended to: - Explicitly pick an instrumentation name via `name = ".."` to distinguish between e.g. `run` of different steps. - Take care to not cause diverging behavior via tracing, e.g. building extra things only when tracing infra is enabled. -### Profiling bootstrap - -You can set the `BOOTSTRAP_PROFILE=1` environment variable to enable command execution profiling during bootstrap. This generates: - -* A Chrome trace file (for visualization in `chrome://tracing` or [Perfetto](https://ui.perfetto.dev)) if tracing is enabled via `BOOTSTRAP_TRACING=COMMAND=trace` -* A plain-text summary file, `bootstrap-profile-{pid}.txt`, listing all commands sorted by execution time (slowest first), along with cache hits and working directories - -Note: the `.txt` report is always generated when `BOOTSTRAP_PROFILE=1` is set — tracing is not required. - -Example usage: - -```bash -$ BOOTSTRAP_PROFILE=1 BOOTSTRAP_TRACING=COMMAND=trace ./x build library -``` - ### rust-analyzer integration? Unfortunately, because bootstrap is a `rust-analyzer.linkedProjects`, you can't ask r-a to check/build bootstrap itself with `tracing` feature enabled to get relevant completions, due to lack of support as described in . diff --git a/src/building/optimized-build.md b/src/building/optimized-build.md index 863ed9749..46def66d1 100644 --- a/src/building/optimized-build.md +++ b/src/building/optimized-build.md @@ -118,7 +118,7 @@ Here is an example of how can `opt-dist` be used locally (outside of CI): ``` 3. Run the tool with the `local` mode and provide necessary parameters: ```bash - ./build/host/stage0-tools-bin/opt-dist local \ + ./build/host/stage1-tools-bin/opt-dist local \ --target-triple \ # select target, e.g. "x86_64-unknown-linux-gnu" --checkout-dir \ # path to rust checkout, e.g. "." --llvm-dir \ # path to built LLVM toolchain, e.g. "/foo/bar/llvm/install" diff --git a/src/stability.md b/src/stability.md index c26d34273..3c4c65fdd 100644 --- a/src/stability.md +++ b/src/stability.md @@ -182,6 +182,11 @@ of the standard library raises it to a warning with `#![warn(deprecated_in_future)]`. ## unstable_feature_bound -The `#[unstable_feature_bound(foo)]` attribute can be used together with `#[unstable]` attribute to mark an `impl` of stable type and stable trait as unstable. In std/core, an item annotated with `#[unstable_feature_bound(foo)]` can only be used by another item that is also annotated with `#[unstable_feature_bound(foo)]`. Outside of std/core, using an item with `#[unstable_feature_bound(foo)]` requires the feature to be enabled with `#![feature(foo)]` attribute on the crate. Currently, only `impl`s and free functions can be annotated with `#[unstable_feature_bound]`. +The `#[unstable_feature_bound(foo)]` attribute can be used together with `#[unstable]` attribute to mark an `impl` of stable type and stable trait as unstable. In std/core, an item annotated with `#[unstable_feature_bound(foo)]` can only be used by another item that is also annotated with `#[unstable_feature_bound(foo)]`. Outside of std/core, using an item with `#[unstable_feature_bound(foo)]` requires the feature to be enabled with `#![feature(foo)]` attribute on the crate. + +Currently, the items that can be annotated with `#[unstable_feature_bound]` are: +- `impl` +- free function +- trait [blog]: https://www.ralfj.de/blog/2018/07/19/const.html