Make rust-toolchain.toml the single source of truth for rustc commit

Replace the hardcoded commit hash in CI and manual checkout steps with
automated reads from rust-toolchain.toml's [metadata] section. Both
local test scripts and CI now use yq to parse the commit, and the
scripts handle bare repos (via worktrees) and regular clones.

Also converts lessons-learned.md into ADR-001 (compat layer decision).

Author: cds-amal

.github/workflows/test.yml

@@ -99,11 +99,25 @@ jobs:
           ref: ${{ github.event.pull_request.head.sha }}
           submodules: recursive
 
+      - name: 'Install yq'
+        uses: mikefarah/yq-action@v4
+
+      - name: 'Read rustc commit from rust-toolchain.toml'
+        id: rustc-meta
+        run: |
+          set -euo pipefail
+          COMMIT=$(yq -r '.metadata.rustc-commit' rust-toolchain.toml)
+          if [ -z "$COMMIT" ] || [ "$COMMIT" = "null" ]; then
+            echo "::error::metadata.rustc-commit not found in rust-toolchain.toml"
+            exit 1
+          fi
+          echo "rustc-commit=$COMMIT" >> "$GITHUB_OUTPUT"
+
       - name: 'Check out Rust repo'
         uses: actions/checkout@v4
         with:
           repository: rust-lang/rust
-          ref: a2545fd6fc66b4323f555223a860c451885d1d2b # hash of Hardcoded Rust version
+          ref: ${{ steps.rustc-meta.outputs.rustc-commit }}
           path: rust
           fetch-depth: 1
 

docs/adr/pr-001-compat-layer-for-rustc-internals.md

@@ -0,0 +1,52 @@
+# ADR-001: Compatibility layer for rustc internal APIs
+
+**Status:** Accepted
+**Date:** 2025-02-21
+
+## Context
+
+stable-mir-json hooks into rustc's internal APIs (`rustc_middle`, `rustc_smir`, `rustc_span`, etc.) to extract MIR data. These APIs are unstable; they change regularly across nightly releases, and the crate names themselves get renamed (the `stable_mir` crate became `rustc_public`, `rustc_smir` became `rustc_public_bridge`, etc.). Before this decision, rustc internals were used directly throughout the codebase: `printer.rs`, `mk_graph/`, `driver.rs`, and various helpers all had their own `extern crate` declarations and direct imports. So a toolchain bump meant hunting through every file that touched a changed API; not fun, and easy to miss things.
+
+## Decision
+
+Route all rustc internal API usage through a single `src/compat/` module. The module re-exports crate names (so a rename like `stable_mir` to `rustc_public` is a one-line alias change in `compat/mod.rs`) and wraps unstable functions behind stable signatures (so a changed calling convention is absorbed in one place).
+
+The compat layer does *not* try to abstract over stable MIR's own public API. When `stable_mir` (the public, downstream-facing API) changes its types, any consumer has to adapt; that's by design. The boundary is: if it's a rustc implementation detail, it goes through compat; if it's the stable MIR contract, it flows through directly.
+
+`src/driver.rs` is the one exception; it uses `rustc_driver` and `rustc_interface` directly because it *is* the rustc integration point. Everything else goes through compat.
+
+## Consequences
+
+**What the compat layer absorbs (rustc internals):**
+
+| Change | Absorbed in |
+|--------|-------------|
+| `collect_and_partition_mono_items` API changes | `compat/mono_collect.rs` |
+| `RunCompiler::new().run()` becoming `run_compiler()` | `driver.rs` |
+| `stable_mir` renamed to `rustc_public` | `compat/mod.rs` (re-exported as alias) |
+| `rustc_smir` renamed to `rustc_public_bridge` | `compat/mod.rs`, `driver.rs` |
+| `IndexedVal` trait moving between crates | `compat/mod.rs` (re-exported) |
+| `FileNameDisplayPreference` variants changing | `compat/spans.rs` |
+
+None of these changes leaked into `printer.rs` or `mk_graph/`. The abstraction worked as designed.
+
+**What still propagates (stable MIR public API evolution):**
+
+- `Rvalue::AddressOf` changed from `Mutability` to `RawPtrKind`
+- `StatementKind::Deinit` and `Rvalue::NullaryOp` removed
+- `AggregateKind::CoroutineClosure` added
+- `Coroutine` and `Dynamic` field count changes
+- `Ty::visit()` return type changed from `()` to `ControlFlow<T>`
+
+These affect `printer.rs` and `mk_graph/` regardless of the compat layer. Any consumer of stable MIR would need to handle them; there's nothing we can (or should) do about that.
+
+**The mk_graph gap (now fixed).** Turns out the `mk_graph/` files originally declared their own `extern crate stable_mir`, bypassing the abstraction entirely. This was introduced in commit `e9395d9` (PR #111) before the compat layer existed; it wasn't an oversight so much as a timing issue. The 13-month toolchain bump exposed the cost: when `stable_mir` was renamed to `rustc_public`, all 5 mk_graph files needed updating, while `printer.rs` needed zero import changes because it already went through compat. Commit `307dcb8` closed this gap by routing all mk_graph imports through `use crate::compat::stable_mir`.
+
+## Validation
+
+We stress-tested the abstraction against two toolchain bumps to see if it actually holds up in practice:
+
+- **6-month jump** (nightly-2024-11-29 to nightly-2025-06-01, rustc 1.85 to 1.89): all internal API changes contained in `compat/` and `driver.rs`
+- **13-month jump** (nightly-2024-11-29 to nightly-2026-01-15, rustc 1.85 to 1.94): same containment, plus the major `stable_mir` to `rustc_public` crate rename absorbed by a single alias in `compat/mod.rs`
+
+Both branches compile and are available for reference: `spike/toolchain-2025-06` and `spike/toolchain-2026-01`, each with a detailed `rustc-<version>.md` breakdown.

lessons-learned.md

@@ -4,7 +4,6 @@ components = ["llvm-tools", "rustc-dev", "rust-src", "rust-analyzer"]
 
 # Ignored by rustup; used by our test scripts.
 # This is the rustc commit that backs the nightly above.
-# UI tests (make test-ui, make remake-ui-tests) need a rust compiler
-# checkout at this commit: git -C $RUST_DIR_ROOT checkout <commit>
+# UI test scripts automatically checkout this commit in RUST_DIR_ROOT.
 [metadata]
 rustc-commit = "a2545fd6fc66b4323f555223a860c451885d1d2b"

rust-toolchain.toml

@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+#
+# Ensures a rust checkout (regular or bare+worktree) is at the commit
+# specified in rust-toolchain.toml's [metadata] rustc-commit field.
+#
+# Usage: source this script after setting RUST_DIR to the repo root.
+# It sets RUST_SRC_DIR to the directory containing the source files
+# (which may differ from RUST_DIR if a worktree is created).
+
+set -u
+
+: "${RUST_DIR:?RUST_DIR must be set before sourcing ensure_rustc_commit.sh}"
+
+# Require yq (mikefarah/yq) for TOML parsing
+if ! command -v yq &> /dev/null; then
+    echo "Error: yq is required but not installed."
+    echo "Install via: brew install yq | apt install yq | nix shell nixpkgs#yq-go"
+    echo "See: https://github.com/mikefarah/yq#install"
+    exit 1
+fi
+
+_SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+_REPO_ROOT=$( cd -- "$_SCRIPT_DIR/../.." &> /dev/null && pwd )
+
+# Read the expected rustc commit from rust-toolchain.toml
+RUSTC_COMMIT=$(yq -r '.metadata.rustc-commit' "$_REPO_ROOT/rust-toolchain.toml")
+if [ -z "$RUSTC_COMMIT" ] || [ "$RUSTC_COMMIT" = "null" ]; then
+    echo "Error: Could not read metadata.rustc-commit from $_REPO_ROOT/rust-toolchain.toml"
+    exit 1
+fi
+
+SHORT_COMMIT="${RUSTC_COMMIT:0:12}"
+
+# Detect whether RUST_DIR is a bare repo
+IS_BARE=$(git -C "$RUST_DIR" rev-parse --is-bare-repository 2>/dev/null)
+
+if [ "$IS_BARE" = "true" ]; then
+    # Bare repo: use worktrees. Check if one already exists at this commit.
+    WORKTREE_DIR="$RUST_DIR/$SHORT_COMMIT"
+
+    if [ -d "$WORKTREE_DIR" ]; then
+        WORKTREE_COMMIT=$(git -C "$WORKTREE_DIR" rev-parse HEAD 2>/dev/null)
+        if [ "${WORKTREE_COMMIT}" = "${RUSTC_COMMIT}" ]; then
+            echo "Worktree already exists at ${WORKTREE_DIR} (${SHORT_COMMIT})"
+            RUST_SRC_DIR="$WORKTREE_DIR"
+        else
+            echo "Error: Worktree at ${WORKTREE_DIR} is at wrong commit (${WORKTREE_COMMIT})"
+            exit 1
+        fi
+    else
+        echo "Creating worktree at ${WORKTREE_DIR} for commit ${SHORT_COMMIT}..."
+        git -C "$RUST_DIR" worktree add "$WORKTREE_DIR" "$RUSTC_COMMIT" --detach --quiet || {
+            echo "Error: Failed to create worktree for commit ${RUSTC_COMMIT} in ${RUST_DIR}"
+            exit 1
+        }
+        RUST_SRC_DIR="$WORKTREE_DIR"
+    fi
+else
+    # Regular repo: checkout the commit directly
+    CURRENT_COMMIT=$(git -C "$RUST_DIR" rev-parse HEAD 2>/dev/null)
+    if [ "${CURRENT_COMMIT}" != "${RUSTC_COMMIT}" ]; then
+        echo "Checking out rustc commit ${SHORT_COMMIT} in ${RUST_DIR}..."
+        git -C "$RUST_DIR" checkout "$RUSTC_COMMIT" --quiet || {
+            echo "Error: Failed to checkout commit ${RUSTC_COMMIT} in ${RUST_DIR}"
+            exit 1
+        }
+    else
+        echo "Rust checkout already at expected commit ${SHORT_COMMIT}"
+    fi
+    RUST_SRC_DIR="$RUST_DIR"
+fi
+
+export RUST_SRC_DIR

tests/ui/ensure_rustc_commit.sh

@@ -20,6 +20,10 @@ fi
 
 RUST_DIR="$1"
 UI_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+# Ensure the rust checkout is at the expected commit (handles bare repos)
+source "$UI_DIR/ensure_rustc_commit.sh"
+
 UI_SOURCES="${UI_DIR}/ui_sources.txt"
 FAILING_TSV="${UI_DIR}/failing.tsv"
 PASSING_TSV="${UI_DIR}/passing.tsv"
@@ -37,7 +41,7 @@ fi
 
 echo "Running UI tests..."
 while read -r test; do
-    full_path="$RUST_DIR/$test"
+    full_path="$RUST_SRC_DIR/$test"
 
     if [ ! -f "$full_path" ]; then
         echo "Error: Test file '$full_path' not found."

tests/ui/remake_ui_tests.sh

@@ -12,8 +12,12 @@ else
     VERBOSE="$2"
 fi
 
-RUST_DIR_ROOT="$1"
+RUST_DIR="$1"
 UI_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+# Ensure the rust checkout is at the expected commit (handles bare repos)
+source "$UI_DIR/ensure_rustc_commit.sh"
+
 PASSING_TSV="${UI_DIR}/passing.tsv"
 
 KEEP_FILES=${KEEP_FILES:-""}
@@ -29,7 +33,7 @@ passed=0
 total=0
 
 while read -r test; do
-    test_path="${RUST_DIR_ROOT}/${test}"
+    test_path="${RUST_SRC_DIR}/${test}"
     test_name="$(basename "$test" .rs)"
     json_file="${PWD}/${test_name}.smir.json"