build-std

Co-authored-by: Adam Gemmell <[email protected]>

Author: davidtwco

text/0000-build-std/0-introduction.md

@@ -3,6 +3,83 @@
 - RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000)
 - Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000)
 
+<!--
+
+This document is long and has lots of authors, follow these rules to maintain a
+consistent voice and structure:
+
+Writing style:
+
+- Text is wrapped at ~80 characters, except for headings
+
+- Use the passive voice
+ 
+- Items in bullet point lists shouldn't end with a period
+
+- Avoid introducing sections that only include other sections and no written
+  content within them
+
+- In the proposals, write as if the feature has already been accepted and
+  implemented
+
+- Leave a line between each bullet point
+
+- With the exception of table of contents-style sections (in this document and
+  in Appendix I), always use reference-style links
+
+- Use a spell checker and broken link checker
+
+Structure:
+
+- Every header has a reference link defined below it with its anchor. Top-level
+  sections just match the header text. Other sections have a prefix (e.g.
+  "rationale-foo", not "foo")
+
+- Add parentheses with `([?][anchor])` wherever an explanation is justified,
+  linking to the relevant sub-section in the rationale/alternative section. This
+  is not required for unresolved questions and future possibilities
+
+- Each justification/alternative in a section must be included in the bullet
+  list at the bottom of that sub-section, likewise with unresolved questions and
+  future possibilities
+
+- Each future possibility, unresolved question and rationale/alternative must
+  backlink back to the section that links to it
+
+- Rationale/alternatives must be in the order that they are referenced in the
+  text, including in end-of-sub-section lists
+
+- Add anyone who has provided feedback prior to the publication of the RFC to
+  the acknowledgements section
+
+Consistency:
+
+- Ensure that Appendix I is up-to-date after any changes
+
+- Appendix II should only reflect the discussion on the cited sources, rather
+  than the current status quo if it has changed
+
+- build-std should not be in backticks (i.e. not `build-std`)
+
+- "pre-built" should always have a hyphen
+
+- Crate names, Cargo configuration options, compiler flags, file names and
+  environment variables should always be in a backticks (e.g. `build-std`)
+
+- Values passed to compiler flags should be in double quotes (e.g. "compatible")
+
+Git:
+
+- Try to keep each individual change to a single commit and describe that change
+  in the commit message
+  
+  - This makes it easier to review and catch-up
+
+- Keep the first line of commit messages limited to 50 characters and the
+  remaining lines to 74 characters
+
+-->
+
 # Summary
 [summary]: #summary
 
@@ -14,6 +91,14 @@ supported:
 2. Rebuilding the standard library with ABI-modifying flags
 3. Building the standard library for tier three targets
 
+This RFC proposes a handful of changes to Cargo, the compiler and standard
+library with the goal of defining a minimal build-std that has the potential of
+being stabilised:
+
+- Explicitly declaring support for the standard library in target specs
+- Explicit and implicit dependencies on the standard library in `Cargo.toml`
+- Re-building the standard library when the profile or target modifiers change
+
 This RFC is co-authored by [David Wood][davidtwco] and
 [Adam Gemmell][adamgemmell]. To improve the readability of this RFC, it does not
 follow the standard RFC template, while still aiming to capture all of the
@@ -38,6 +123,20 @@ As a general rule, this RFC tries to answer the question "what crates of the
 standard library get built and when do they get built" and considers anything
 else as likely out-of-scope.
 
+### Acknowledgements
+[acknowledgements]: #acknowledgements
+
+This RFC would not have been possible without the advice, feedback and support
+of [Josh Triplett][joshtriplett], [Eric Huss][ehuss],
+[Wesley Wiser][wesleywiser] and [Tomas Sedovic][tomassedovic].
+
+Thanks to [mati865] for advising on some of the specifics related to special
+object files, [petrochenkov] for his expertise on rustc's dependency loading and
+name resolution; [fee1-dead] for their early and thorough reviews and to
+[Ed Page][epage] for writing about opaque dependencies.
+
+Thanks to [Jacob Bramley][jacobbramley] for their feedback on early drafts.
+
 ### Terminology
 [terminology]: #terminology
 
@@ -48,15 +147,47 @@ The following terminology is used throughout the RFC:
 - "std" is used to refer only to the `std` crate, not the entirety of the standard
   library
 
+Throughout the RFC's "Proposal" sections, parentheses with "?" links will be
+present that which link the relevant section in the appropriate "Rationale and
+alternatives" section to justify a decision or provide alternatives to it.
+
+Additionally, "note alerts" will be used in the *Proposal* sections to separate
+implementation considerations from the core proposal. Implementation detail
+should be considered non-normative. These details could change during
+implementation and are present solely to demonstrate that the implementation
+feasibility has been considered and to provide an example of how implementation
+could proceed.
+
+> [!NOTE]
+>
+> This is an example of a "note alert" that will be used to separate
+> implementation detail from the proposal proper.
+
 # Contents
 [contents]: #contents
 
-This RFC has the following contents:
+This RFC has been split into multiple stages. Each stage is a
+self-contained proposal building on the previous, which could be accepted,
+implemented and stabilised on its own.
+
+As build-std is a complex feature with many interdependent design decisions, it
+is challenging to draft a proposal that is small enough to have an achievable
+scope in the short-to-medium term while making a convincing argument that it is
+forward-compatible with any desired future extensions. A staged proposal enables
+this - each stage can have a small and achievable scope, while still allowing a
+reviewer to skip ahead and get a sense of what is planned and how that builds on
+what came before.
+
+Later stages are less detailed and complete than the previous stages, intended
+to indicate the intended direction that the RFC will take and help provide
+context for the proposals of earlier stages.
 
 1. [Summary][summary] (you are here)
 
   - Introduction to the proposal, its scope, terminology/conventions used and
     the structure of the RFC
+
+  - [Proposal-wide rationale and alternatives][rationale-and-alternatives]
 
 2. [Background](./1-background.md)
 
@@ -74,7 +205,75 @@ This RFC has the following contents:
   - Descriptions of the varied problems that build-std has been proposed as a
     solution to
 
-1.  [Appendix II: Exhaustive literature review](./4-appendix-literature-review.md)
+5. [Stage 1a: build-std=always](./4-stage-1a.md)
+
+  - Proposes adding a `build-std = "always|off"` option to the Cargo
+    configuration which will unconditionally re-build the standard library
+    crates listed in the `build-std-crates` option
+
+  - [Proposal](./4-stage-1a.md#proposal)
+
+  - [Rationale and alternatives](./4-stage-1a.md#rationale-and-alternatives)
+
+  - [Unresolved questions](./4-stage-1a.md#unresolved-questions)
+
+  - [Future possibilities](./4-stage-1a.md#future-possibilities)
+
+6. [Stage 1b: Explicit dependencies](./5-stage-1b.md)
+
+  - Proposes supporting explicit dependencies on the standard library crates in
+    `Cargo.toml`
+
+    - Enables Cargo to determine which standard library crates are required by
+      the crate graph without `build-std-crates` being set
+    
+    - Necessary for future extensions which support public/private standard
+      library dependencies or enabling features of the standard library
+
+  - [Proposal](./5-stage-1b.md#proposal)
+
+  - [Rationale and alternatives](./5-stage-1b.md#rationale-and-alternatives)
+
+  - [Unresolved questions](./5-stage-1b.md#unresolved-questions)
+
+  - [Future possibilities](./5-stage-1b.md#future-possibilities)
+
+7. [Stage 2: build-std=compatible](./6-stage-2.md)
+
+  - Proposes extending the `build-std` option with a new `compatible` value
+    which will become the default and automatically rebuilds the standard
+    library when it is necessary to maintain compatibility with the compiler
+    flags used by the rest of the crate graph.
+
+  - [Proposal](./6-stage-2.md#proposal)
+
+  - [Rationale and alternatives](./6-stage-2.md#rationale-and-alternatives)
+
+  - [Unresolved questions](./6-stage-2.md#unresolved-questions)
+
+  - [Future possibilities](./6-stage-2.md#future-possibilities)
+
+8.  [Stage 3: build-std=match-profile](./7-stage-3.md)
+
+  - Proposes extending the `build-std` option with new values which
+    automatically rebuild the standard library to match the user's current
+    profile.
+
+  - [Proposal](./7-stage-3.md#proposal)
+
+  - [Rationale and alternatives](./7-stage-3.md#rationale-and-alternatives)
+
+  - [Unresolved questions](./7-stage-3.md#unresolved-questions)
+
+  - [Future possibilities](./7-stage-3.md#future-possibilities)
+
+9.  [Appendix I: Summary of changes](./8-appendix-summary-of-changes.md)
+    
+  - Summary of each of the changes from each stage which would need implemented
+    in the Rust toolchain, grouped by the project team whose purview the change
+    would fall under
+
+10.  [Appendix II: Exhaustive literature review](./9-appendix-literature-review.md)
 
   - More detailed summaries of the relevant issues, discussions, pull requests
     and proposals that comprise the history of the build-std feature since
@@ -83,5 +282,54 @@ This RFC has the following contents:
   - [*History*](./2-history.md) aims to summarise this content further and
     cover everything that should be necessary to understand the proposal
 
+# Rationale and alternatives
+[rationale-and-alternatives]: #rationale-and-alternatives
+
+These rationales and alternatives apply to the proposal as-a-whole, rather than
+any specific stage:
+
+## Why not do nothing?
+[rationale-why-not-do-nothing]: #why-not-do-nothing
+
+Support for rebuilding the standard library is a long-standing feature request
+from subsets of the Rust community and blocks the work of some project teams
+(e.g. sanitisers and branch protection in the compiler team, amongst others).
+Inaction forces these users to remain on nightly and depend on the unstable
+`-Zbuild-std` flag indefinitely. RFCs and discussion dating back to the first
+stable release of the language demonstrate the longevity of build-std as a
+need.
+
+## Shouldn't build-std be part of rustup?
+[rationale-in-rustup]: #shouldnt-build-std-be-part-of-rustup
+
+build-std is effectively creating a new sysroot with a customised standard
+library. rustup as Rust's toolchain manager has lots of existing machinery
+to create and maintain sysroots. rustup knows how to download `rust-src`, it
+knows how to create a new toolchain from an existing sysroot (as in
+`rustup toolchain link`), it would only need to learn how to invoke Cargo on the
+`rust-src` sources. rustup would be invoking tools from the next layer of
+abstraction (Cargo) in the same way that Cargo invokes tools from the layer of
+abstraction after it (rustc).
+
+A brief prototype of this idea was created and a
+[short design document was drafted][why-not-rustup] before concluding that it
+would not be possible. With artifact dependencies, it may be desirable to build
+with a different standard library and if rustup was creating different
+toolchains per-customised standard library then Cargo would need to have
+knowledge of these to switch between them, which isn't possible (and something
+of a layering violation). It is also unclear how Cargo would find and use the
+uncustomized host sysroot for build scripts and procedural macros.
+
 [davidtwco]: https://github.com/davidtwco
-[adamgemmell]: https://github.com/adamgemmell
+[adamgemmell]: https://github.com/adamgemmell
+[ehuss]: https://github.com/ehuss
+[epage]: https://github.com/epage
+[fee1-dead]: https://github.com/fee1-dead
+[jacobbramley]: https://github.com/jacobbramley
+[joshtriplett]: https://github.com/joshtriplett
+[mati865]: https://github.com/mati865
+[petrochenkov]: https://github.com/petrochenkov
+[tomassedovic]: https://github.com/tomassedovic
+[wesleywiser]: https://github.com/wesleywiser
+
+[why-not-rustup]: https://hackmd.io/@davidtwco/rkYRlKv_1x

text/0000-build-std/2-history.md

@@ -456,10 +456,10 @@ features that are related or would be beneficial for build-std:
       are unlikely to be needed during debugging
 
 [motivation]: ./3-motivation.md
-[appendix-ii]: ./4-appendix-literature-review.md
-[appendix-ii-impl]: ./4-appendix-literature-review.md#implementation
-[appendix-ii-bugs]: ./4-appendix-literature-review.md#bugs-in-the-compiler-or-standard-library
-[appendix-ii-cargo-feats]: ./4-appendix-literature-review.md#cargo-feature-requests-narrowly-applied-to-build-std
+[appendix-ii]: ./9-appendix-literature-review.md
+[appendix-ii-impl]: ./9-appendix-literature-review.md#implementation
+[appendix-ii-bugs]: ./9-appendix-literature-review.md#bugs-in-the-compiler-or-standard-library
+[appendix-ii-cargo-feats]: ./9-appendix-literature-review.md#cargo-feature-requests-narrowly-applied-to-build-std
 
 [JOSH]: https://josh-project.github.io/josh/intro.html
 [rust-lang/cargo]: https://github.com/rust-lang/cargo

text/0000-build-std/3-motivation.md

@@ -5,6 +5,8 @@ While the pre-built standard library has been sufficient for the majority of
 Rust users, there are a variety of use-cases which require the ability to
 re-build the standard library.
 
+This RFC aims to support the following use cases:
+
 1. **Re-building the standard library with different codegen flags or profile**
    ([wg-cargo-std-aware#2])
 
@@ -62,21 +64,29 @@ re-build the standard library.
     that would not be appropriate for the pre-built standard library, so is forced
     to require nightly and build its own sysroot
 
-7. **Using the standard library with custom targets**
+The following use cases are not supported by this RFC, but could be supported
+with follow-up RFCs (and this RFC will attempt to ensure they remain viable as
+future possibilities):
+
+1. **Using the standard library with custom targets**
 
   - There is no stable mechanism for using the standard library for a custom
     target (using target-spec-json)
   - Like tier three targets, these targets often only support `core` and are
     forced to use nightly today
 
-8. **Enabling Cargo features for the standard library** ([wg-cargo-std-aware#4])
+2. **Enabling Cargo features for the standard library** ([wg-cargo-std-aware#4])
 
   - There are opportunities to expose Cargo features from the standard library that
     would be useful for certain subsets of the Rust users.
     - For example, embedded users may want to enable a feature like `optimize_for_size` or
       `panic_immediate_abort` to reduce binary size
 
-9. **Modifying the source code of the standard library** ([wg-cargo-std-aware#7])
+Some use cases are unlikely to supported by the project unless a new and
+compelling use-case is presented, and so this RFC may make decisions which make
+these motivations harder to solve in future:
+
+1. **Modifying the source code of the standard library** ([wg-cargo-std-aware#7])
 
   - Some platforms require a heavily modified standard library that would not
     be suitable for upstreaming, such as [Apache's SGX SDK][sgx] which replaces