From 912575e174f41c19e56a9b34d0ec8a81e8775373 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Tue, 11 Feb 2025 13:21:40 +0100 Subject: [PATCH 1/9] Update documentation --- src/building/optimized-build.md | 2 +- src/tests/ci.md | 6 +++--- src/tests/docker.md | 9 +++++++++ 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/src/building/optimized-build.md b/src/building/optimized-build.md index 8feda5982..f8ca1d0dc 100644 --- a/src/building/optimized-build.md +++ b/src/building/optimized-build.md @@ -126,4 +126,4 @@ Here is an example of how can `opt-dist` be used locally (outside of CI): [`Environment`]: https://github.com/rust-lang/rust/blob/ee451f8faccf3050c76cdcd82543c917b40c7962/src/tools/opt-dist/src/environment.rs#L5 > Note: if you want to run the actual CI pipeline, instead of running `opt-dist` locally, -> you can execute `python3 src/ci/github-actions/ci.py run-local dist-x86_64-linux`. +> you can execute `cargo run --manifest-path src/ci/citool/Cargo.toml run-local dist-x86_64-linux`. diff --git a/src/tests/ci.md b/src/tests/ci.md index a4b22392f..ae6adb678 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -28,7 +28,7 @@ Our CI is primarily executed on [GitHub Actions], with a single workflow defined in [`.github/workflows/ci.yml`], which contains a bunch of steps that are unified for all CI jobs that we execute. When a commit is pushed to a corresponding branch or a PR, the workflow executes the -[`src/ci/github-actions/ci.py`] script, which dynamically generates the specific CI +[`src/ci/citool`] crate, which dynamically generates the specific CI jobs that should be executed. This script uses the [`jobs.yml`] file as an input, which contains a declarative configuration of all our CI jobs. @@ -299,7 +299,7 @@ platform’s custom [Docker container]. This has a lot of advantages for us: - We can avoid reinstalling tools (like QEMU or the Android emulator) every time thanks to Docker image caching. - Users can run the same tests in the same environment locally by just running - `python3 src/ci/github-actions/ci.py run-local `, which is awesome to debug failures. Note that there are only linux docker images available locally due to licensing and + `cargo run --manifest-path src/ci/citool/Cargo.toml run-local `, which is awesome to debug failures. Note that there are only linux docker images available locally due to licensing and other restrictions. The docker images prefixed with `dist-` are used for building artifacts while @@ -443,7 +443,7 @@ this: [GitHub Actions]: https://github.com/rust-lang/rust/actions [`jobs.yml`]: https://github.com/rust-lang/rust/blob/master/src/ci/github-actions/jobs.yml [`.github/workflows/ci.yml`]: https://github.com/rust-lang/rust/blob/master/.github/workflows/ci.yml -[`src/ci/github-actions/ci.py`]: https://github.com/rust-lang/rust/blob/master/src/ci/github-actions/ci.py +[`src/ci/citool`]: https://github.com/rust-lang/rust/blob/master/src/ci/citool [rust-lang-ci]: https://github.com/rust-lang-ci/rust/actions [bors]: https://github.com/bors [homu]: https://github.com/rust-lang/homu diff --git a/src/tests/docker.md b/src/tests/docker.md index 2ca08d421..a8a388ef9 100644 --- a/src/tests/docker.md +++ b/src/tests/docker.md @@ -53,6 +53,15 @@ Some additional notes about using the interactive mode: containers. With the container name, run `docker exec -it /bin/bash` where `` is the container name like `4ba195e95cef`. +The approach described above is a relatively low-level interface for running the Docker images +directly. If you want to run a full CI Linux job locally with Docker, in a way that is as close to CI as possible, you can use the following command: + +```bash +cargo run --manifest-path src/ci/citool/Cargo.toml run-local +# For example: +cargo run --manifest-path src/ci/citool/Cargo.toml run-local dist-x86_64-linux-alt +``` + [Docker]: https://www.docker.com/ [`src/ci/docker`]: https://github.com/rust-lang/rust/tree/master/src/ci/docker [`src/ci/docker/run.sh`]: https://github.com/rust-lang/rust/blob/master/src/ci/docker/run.sh From 8044303cbfaca1ca145bb8002d478af970e0835a Mon Sep 17 00:00:00 2001 From: Noratrieb <48135649+Noratrieb@users.noreply.github.com> Date: Sat, 18 Jan 2025 18:19:42 +0100 Subject: [PATCH 2/9] Support raw-dylib link kind on ELF raw-dylib is a link kind that allows rustc to link against a library without having any library files present. This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker. While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future. This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time. --- src/tests/directives.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tests/directives.md b/src/tests/directives.md index 00bb2bc4d..af9cb85d2 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -142,6 +142,7 @@ Some examples of `X` in `ignore-X` or `only-X`: matches that target as well as the emscripten targets. - Pointer width: `32bit`, `64bit` - Endianness: `endian-big` +- Binary format: `elf` - Stage: `stage0`, `stage1`, `stage2` - Channel: `stable`, `beta` - When cross compiling: `cross-compile` From 6a61f6f721c0e1999a04ac0f7a3f37af5a47eb34 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B8=E6=9D=B0=E5=8F=8B=20Jieyou=20Xu=20=28Joe=29?= <39484203+jieyouxu@users.noreply.github.com> Date: Wed, 5 Feb 2025 21:40:57 +0800 Subject: [PATCH 3/9] rustc-dev-guide: remove mentions of legacy `Makefile` run-make infra And remove outdated requirements to run `run-make` tests on Windows. --- src/tests/compiletest.md | 30 +----------------------------- src/tests/directives.md | 7 +------ src/tests/running.md | 25 ------------------------- 3 files changed, 2 insertions(+), 60 deletions(-) diff --git a/src/tests/compiletest.md b/src/tests/compiletest.md index 459c08290..2905e470f 100644 --- a/src/tests/compiletest.md +++ b/src/tests/compiletest.md @@ -74,8 +74,7 @@ The following test suites are available, with links for more information: ### General purpose test suite -[`run-make`](#run-make-tests) are general purpose tests using Rust programs (or -Makefiles (legacy)). +[`run-make`](#run-make-tests) are general purpose tests using Rust programs. ### Rustdoc test suites @@ -396,14 +395,6 @@ your test, causing separate files to be generated for 32bit and 64bit systems. ### `run-make` tests -> **Note on phasing out `Makefile`s** -> -> We are planning to migrate all existing Makefile-based `run-make` tests -> to Rust programs. You should not be adding new Makefile-based `run-make` -> tests. -> -> See . - The tests in [`tests/run-make`] are general-purpose tests using Rust *recipes*, which are small programs (`rmake.rs`) allowing arbitrary Rust code such as `rustc` invocations, and is supported by a [`run_make_support`] library. Using @@ -424,11 +415,6 @@ Compiletest directives like `//@ only-` or `//@ ignore-` are supported in `rmake.rs`, like in UI tests. However, revisions or building auxiliary via directives are not currently supported. -Two `run-make` tests are ported over to Rust recipes as examples: - -- -- - #### Quickly check if `rmake.rs` tests can be compiled You can quickly check if `rmake.rs` tests can be compiled without having to @@ -481,20 +467,6 @@ Then add a corresponding entry to `"rust-analyzer.linkedProjects"` ], ``` -#### Using Makefiles (legacy) - -
-You should avoid writing new Makefile-based `run-make` tests. -
- -Each test should be in a separate directory with a `Makefile` indicating the -commands to run. - -There is a [`tools.mk`] Makefile which you can include which provides a bunch of -utilities to make it easier to run commands and compare outputs. Take a look at -some of the other tests for some examples on how to get started. - -[`tools.mk`]: https://github.com/rust-lang/rust/blob/master/tests/run-make/tools.mk [`tests/run-make`]: https://github.com/rust-lang/rust/tree/master/tests/run-make [`run_make_support`]: https://github.com/rust-lang/rust/tree/master/src/tools/run-make-support diff --git a/src/tests/directives.md b/src/tests/directives.md index 00bb2bc4d..c72c7d7d8 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -6,10 +6,7 @@ FIXME(jieyouxu) completely revise this chapter. --> -Directives are special comments that tell compiletest how to build and interpret -a test. They must appear before the Rust source in the test. They may also -appear in `rmake.rs` or legacy Makefiles for [run-make -tests](compiletest.md#run-make-tests). +Directives are special comments that tell compiletest how to build and interpret a test. They must appear before the Rust source in the test. They may also appear in `rmake.rs` [run-make tests](compiletest.md#run-make-tests). They are normally put after the short comment that explains the point of this test. Compiletest test suites use `//@` to signal that a comment is a directive. @@ -221,8 +218,6 @@ The following directives will check LLVM support: [`aarch64-gnu-debug`]), which only runs a subset of `run-make` tests. Other tests with this directive will not run at all, which is usually not what you want. - - Notably, the [`aarch64-gnu-debug`] CI job *currently* only runs `run-make` - tests which additionally contain `clang` in their test name. See also [Debuginfo tests](compiletest.md#debuginfo-tests) for directives for ignoring debuggers. diff --git a/src/tests/running.md b/src/tests/running.md index 6ce650923..9ddf0afee 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -238,30 +238,6 @@ This is much faster, but doesn't always work. For example, some tests include directives that specify specific compiler flags, or which rely on other crates, and they may not run the same without those options. -## Running `run-make` tests - -### Windows - -Running the `run-make` test suite on Windows is a currently bit more involved. -There are numerous prerequisites and environmental requirements: - -- Install msys2: -- Specify `MSYS2_PATH_TYPE=inherit` in `msys2.ini` in the msys2 installation directory, run the - following with `MSYS2 MSYS`: - - `pacman -Syuu` - - `pacman -S make` - - `pacman -S diffutils` - - `pacman -S binutils` - - `./x test run-make` (`./x test tests/run-make` doesn't work) - -There is [on-going work][port-run-make] to not rely on `Makefile`s in the -run-make test suite. Once this work is completed, you can run the entire -`run-make` test suite on native Windows inside `cmd` or `PowerShell` without -needing to install and use MSYS2. As of Oct 2024, it is -already possible to run the vast majority of the `run-make` test suite outside -of MSYS2, but there will be failures for the tests that still use `Makefile`s -due to not finding `make`. - ## Running tests on a remote machine Tests may be run on a remote machine (e.g. to test builds for a different @@ -406,4 +382,3 @@ If you encounter bugs or problems, don't hesitate to open issues on the repository](https://github.com/rust-lang/rustc_codegen_gcc/). [`tests/ui`]: https://github.com/rust-lang/rust/tree/master/tests/ui -[port-run-make]: https://github.com/rust-lang/rust/issues/121876 From 4801165af51ba8fde444d0821002b6e99f3e4a9a Mon Sep 17 00:00:00 2001 From: Zalathar Date: Mon, 3 Mar 2025 20:17:26 +1100 Subject: [PATCH 4/9] Remove some unnecessary aliases from `rustc_data_structures::sync` With the removal of `cfg(parallel_compiler)`, these are always shared references and `std::sync::OnceLock`. --- src/parallel-rustc.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/parallel-rustc.md b/src/parallel-rustc.md index c5b70706a..690fb19c9 100644 --- a/src/parallel-rustc.md +++ b/src/parallel-rustc.md @@ -46,10 +46,8 @@ are implemented differently depending on whether `parallel-compiler` is true. | data structure | parallel | non-parallel | | -------------------------------- | --------------------------------------------------- | ------------ | -| OnceCell | std::sync::OnceLock | std::cell::OnceCell | | Lock\ | (parking_lot::Mutex\) | (std::cell::RefCell) | | RwLock\ | (parking_lot::RwLock\) | (std::cell::RefCell) | -| MTRef<'a, T> | &'a T | &'a mut T | | MTLock\ | (Lock\) | (T) | | ReadGuard | parking_lot::RwLockReadGuard | std::cell::Ref | | MappedReadGuard | parking_lot::MappedRwLockReadGuard | std::cell::Ref | From 93eca2b4945a7e03cf5833bb432882e12f8fe758 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B8=E6=9D=B0=E5=8F=8B=20Jieyou=20Xu=20=28Joe=29?= <39484203+jieyouxu@users.noreply.github.com> Date: Mon, 24 Feb 2025 20:14:25 +0800 Subject: [PATCH 5/9] Document that `rmake.rs`/`run-make-support` may not use unstable features --- src/tests/compiletest.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/tests/compiletest.md b/src/tests/compiletest.md index 2905e470f..a6996e398 100644 --- a/src/tests/compiletest.md +++ b/src/tests/compiletest.md @@ -415,6 +415,10 @@ Compiletest directives like `//@ only-` or `//@ ignore-` are supported in `rmake.rs`, like in UI tests. However, revisions or building auxiliary via directives are not currently supported. +`rmake.rs` and `run-make-support` may *not* use any nightly/unstable features, +as they must be compilable by a stage 0 rustc that may be a beta or even stable +rustc. + #### Quickly check if `rmake.rs` tests can be compiled You can quickly check if `rmake.rs` tests can be compiled without having to From 074e4787bdb0d8fd76398fb87dfc6f34746ab4f9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Mon, 10 Mar 2025 13:35:50 +0100 Subject: [PATCH 6/9] Modify try-job documentation --- src/tests/ci.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/tests/ci.md b/src/tests/ci.md index ae6adb678..7fd4c1697 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -134,15 +134,17 @@ There are several use-cases for try builds: the [dist-x86_64-linux] CI job. - Run a specific CI job (e.g. Windows tests) on a PR, to quickly test if it passes the test suite executed by that job. You can select which CI jobs will - be executed in the try build by adding up to 10 lines containing `try-job: - ` to the PR description. All such specified jobs will be executed + be executed in the try build by adding lines containing `try-job: + ` to the PR description. All such specified jobs will be executed in the try build once the `@bors try` command is used on the PR. If no try jobs are specified in this way, the jobs defined in the `try` section of - [`jobs.yml`] will be executed by default. + [`jobs.yml`] will be executed by default. Each pattern can either be an exact + name of a job or a glob pattern that matches multiple jobs, for example + `*msvc*` or `*-alt`. You can start at most 20 jobs in a single try build. > **Using `try-job` PR description directives** > -> 1. Identify which set of try-jobs (max 10) you would like to exercise. You can +> 1. Identify which set of try-jobs you would like to exercise. You can > find the name of the CI jobs in [`jobs.yml`]. > > 2. Amend PR description to include (usually at the end of the PR description) @@ -153,9 +155,10 @@ There are several use-cases for try builds: > > try-job: x86_64-msvc > try-job: test-various +> try-job: *-alt > ``` > -> Each `try-job` directive must be on its own line. +> Each `try-job` pattern must be on its own line. > > 3. Run the prescribed try jobs with `@bors try`. As aforementioned, this > requires the user to either (1) have `try` permissions or (2) be delegated From dd0359476d06336734803e1c081b17a507147708 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Mon, 10 Mar 2025 13:56:42 +0100 Subject: [PATCH 7/9] Handle backticks in try job patterns --- src/tests/ci.md | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/src/tests/ci.md b/src/tests/ci.md index 7fd4c1697..0c0f750a4 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -133,29 +133,34 @@ There are several use-cases for try builds: Again, a working compiler build is needed for this, which can be produced by the [dist-x86_64-linux] CI job. - Run a specific CI job (e.g. Windows tests) on a PR, to quickly test if it - passes the test suite executed by that job. You can select which CI jobs will - be executed in the try build by adding lines containing `try-job: - ` to the PR description. All such specified jobs will be executed - in the try build once the `@bors try` command is used on the PR. If no try - jobs are specified in this way, the jobs defined in the `try` section of - [`jobs.yml`] will be executed by default. Each pattern can either be an exact - name of a job or a glob pattern that matches multiple jobs, for example - `*msvc*` or `*-alt`. You can start at most 20 jobs in a single try build. + passes the test suite executed by that job. + +You can select which CI jobs will +be executed in the try build by adding lines containing `try-job: +` to the PR description. All such specified jobs will be executed +in the try build once the `@bors try` command is used on the PR. If no try +jobs are specified in this way, the jobs defined in the `try` section of +[`jobs.yml`] will be executed by default. + +Each pattern can either be an exact name of a job or a glob pattern that matches multiple jobs, +for example `*msvc*` or `*-alt`. You can start at most 20 jobs in a single try build. When using +glob patterns, you might want to wrap them in backticks (`` ` ``) to avoid GitHub rendering +the pattern as Markdown. > **Using `try-job` PR description directives** > > 1. Identify which set of try-jobs you would like to exercise. You can > find the name of the CI jobs in [`jobs.yml`]. > -> 2. Amend PR description to include (usually at the end of the PR description) -> e.g. +> 2. Amend PR description to include a set of patterns (usually at the end +> of the PR description), for example: > > ```text > This PR fixes #123456. > > try-job: x86_64-msvc > try-job: test-various -> try-job: *-alt +> try-job: `*-alt` > ``` > > Each `try-job` pattern must be on its own line. From 7f5b5c36f82be0ff0e270f648adc90f3042b5c2a Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Fri, 21 Feb 2025 18:33:05 +1100 Subject: [PATCH 8/9] Move methods from `Map` to `TyCtxt`, part 4. Continuing the work from #137350. Removes the unused methods: `expect_variant`, `expect_field`, `expect_foreign_item`. Every method gains a `hir_` prefix. --- src/hir.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/hir.md b/src/hir.md index 51893d537..75f5a9e20 100644 --- a/src/hir.md +++ b/src/hir.md @@ -139,12 +139,12 @@ defined in the map. By matching on this, you can find out what sort of node the `HirId` referred to and also get a pointer to the data itself. Often, you know what sort of node `n` is – e.g. if you know that `n` must be some HIR expression, you can do -[`tcx.hir().expect_expr(n)`][expect_expr], which will extract and return the +[`tcx.hir_expect_expr(n)`][expect_expr], which will extract and return the [`&hir::Expr`][Expr], panicking if `n` is not in fact an expression. [find]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.find [`Node`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.Node.html -[expect_expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.expect_expr +[expect_expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.expect_expr [Expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/struct.Expr.html Finally, you can use the HIR map to find the parents of nodes, via From 2aa579f75233f3e63e3ffa8ee51ab5a362e93f8b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B8=E6=9D=B0=E5=8F=8B=20Jieyou=20Xu=20=28Joe=29?= <39484203+jieyouxu@users.noreply.github.com> Date: Thu, 13 Mar 2025 15:19:22 +0800 Subject: [PATCH 9/9] Preparing for merge from rustc --- rust-version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust-version b/rust-version index ce21bb8ef..eb779d9ab 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -4ecd70ddd1039a3954056c1071e40278048476fa +8536f201ffdb2c24925d7f9e87996d7dca93428b