You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: text/0000-mergeable-rustdoc-cross-crate-info.md
+10-10Lines changed: 10 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,7 +21,7 @@ There are some files in the rustdoc output directory that are read and overwritt
21
21
22
22
Build systems may run build actions in a distributed environment across separate logical filesystems. It might also be desirable to run rustdoc in a lock-free parallel mode, where every rustdoc process writes to a disjoint set of files.
23
23
24
-
Cargo fully supports cross-crate information, at the cost of requiring read-write access to the documentation root (`target/doc/`). There are significant scalability issues with this approach.
24
+
Cargo fully supports cross-crate information, at the cost of requiring global read-write access to the doc root (`target/doc`). There are significant scalability issues with this approach.
25
25
26
26
Rustdoc needing global mutable access to the files that encode this cross-crate information has implications for caching, reproducible builds, and content hashing. By adding an option to avoid this mutation, rustdoc will serve as a first-class citizen in non-cargo build systems.
27
27
@@ -287,14 +287,14 @@ $ tree . -a
287
287
288
288
Currently, cross-crate information is written during the invocation of the `write_shared` function in [write_shared.rs](https://github.com/rust-lang/rust/blob/04ab7b2be0db3e6787f5303285c6b2ee6279868d/src/librustdoc/html/render/write_shared.rs#L47). This proposal does not add any new CCI or change their contents (modulo sorting order, whitespace).
289
289
290
-
The existing cross-crate information files, like `search-index.js`, all are lists of elements, rendered in an specified way (e.g. as a JavaScript file with a JSON array or an HTML index page containing an unordered list). The current rustdoc (in `write_shared`) pushes the current crate's version of the CCI into the one that is already found in `doc/`, and renders a new version. The rest of the proposal uses the term **part** to refer to the pre-merged, pre-rendered element of a the CCI.
290
+
The existing cross-crate information files, like `search-index.js`, all are lists of elements, rendered in an specified way (e.g. as a JavaScript file with a JSON array or an HTML index page containing an unordered list). The current rustdoc (in `write_shared`) pushes the current crate's version of the CCI into the one that is already found in `doc`, and renders a new version. The rest of the proposal uses the term **part** to refer to the pre-merged, pre-rendered element of a the CCI.
291
291
292
292
## New subdirectory: `<parts out dir>/<crate name>/<cci type>`
293
293
294
294
Typically, `<parts out dir>` is selected as `./target/doc.parts`. The `<parts out dir>/<crate name>/<cci type>` files contain the unmerged contents of a single crates' version of their corresponding CCI. It is written if the flag `--parts-out-dir=<parts out dir>` is provided.
295
295
296
296
Every file in `<parts out dir>/<crate name>/*` is a JSON array. Every element of the
297
-
array is a two-element array: a destination filename (relative to `doc/`), and
297
+
array is a two-element array: a destination filename (relative to the doc root), and
298
298
the representation of the part. The representation of that part depends on the type
299
299
of CCI that it describes.
300
300
@@ -358,7 +358,7 @@ In the Guide-level explanation, for example, crate `i` needs to identify the loc
358
358
359
359
## Merge step
360
360
361
-
This step is provided with a list of crates. It merges their documentation. This step involves copying parts (individual item, module documentation) from each of the provided crates. It merges the parts, renders, and writes the CCI to the documentation root.
361
+
This step is provided with a list of crates. It merges their documentation. This step involves copying parts (individual item, module documentation) from each of the provided crates. It merges the parts, renders, and writes the CCI to the doc root.
362
362
363
363
Discussion of the merge step is described in the Unresolved questions.
364
364
@@ -368,7 +368,7 @@ The WIP may change the sorting order of the elements in the CCI. It does not cha
368
368
369
369
# Rationale and alternatives
370
370
371
-
Running rustdoc in parallel is essential in enabling the tool to scale to large projects. The alternative, implemented by Cargo, is to run rustdoc in parallel by locking the CCI files. There are some environments where having synchronized access to the CCI is impossible. This proposal implements a reasonable approach to shared rustdoc, because it cleanly enables the addition of new kinds of CCI without changing existing documentation.
371
+
Running rustdoc in parallel is essential in enabling the tool to scale to large projects. The approach implemented by Cargo is to run rustdoc in parallel by locking the CCI files. There are some environments where having synchronized access to the CCI is impossible. This proposal implements a reasonable approach to shared rustdoc, because it cleanly enables the addition of new kinds of CCI without changing existing documentation.
372
372
373
373
# Prior art
374
374
@@ -426,15 +426,15 @@ Require users to generate documentation bundles via an index crate (current) vs.
426
426
427
427
If one would like to merge the documentation of several crates, we could continue to require users to provide an index crate, like [the fuchsia index](https://fuchsia-docs.firebaseapp.com/rust/rustdoc_index/). This serves as the target of the rustdoc invocation, and the landing page for the collected documentation. Supporting only this style of index would require the fewest changes. This is the mode described in the Guide-level explanation.
428
428
429
-
The proposition, to allow users of rustdoc the flexibility of not having to produce an index, is to allow rustdoc to be run in a mode where no target crate is provided. The source crates are provided to rustdoc, through a mechanism like `--extern`, rustdoc merges and writes the CCI, and copies the item and module links to `doc/`. This would require more extensive changes, as rustdoc assumes that it is invoked with a target crate. This mode is somewhat analogous to the [example scraping mode](https://github.com/rust-lang/rfcs/blob/master/text/3123-rustdoc-scrape-examples.md). Having to create an index crate, that actively uses all of the crates in the environment, might prohibit the use of this feature in settings where users do not intend to produce an index, or where exhaustively listing all dependencies (to `--extern` them) is difficult.
429
+
The proposition, to allow users of rustdoc the flexibility of not having to produce an index, is to allow rustdoc to be run in a mode where no target crate is provided. The source crates are provided to rustdoc, through a mechanism like `--extern`, rustdoc merges and writes the CCI, and copies the item and module links to the doc root. This would require more extensive changes, as rustdoc assumes that it is invoked with a target crate. This mode is somewhat analogous to the [example scraping mode](https://github.com/rust-lang/rfcs/blob/master/text/3123-rustdoc-scrape-examples.md). Having to create an index crate, that actively uses all of the crates in the environment, might prohibit the use of this feature in settings where users do not intend to produce an index, or where exhaustively listing all dependencies (to `--extern` them) is difficult.
430
430
431
431
## Unconditionally generating the `doc.parts` files?
432
432
433
433
Generate no extra files (current) vs. unconditionally creating `doc.parts` to enable more complex future CCI (should consider)
434
434
435
435
The current version of rustdoc performs merging by [collecting JSON](https://github.com/rust-lang/rust/blob/c25ac9d6cc285e57e1176dc2da6848b9d0163810/src/librustdoc/html/render/write_shared.rs#L166) blobs from the contents of the already-rendered CCI.
436
436
This proposal proposes to continue reading from the rendered cross-crate information under the default `--read-rendered-cci=true`. It can also read `doc.parts` files, under `--fetch-parts`. However, there are several issues with reading from the rendered CCI that must be stated:
437
-
*If a user has a single `doc/` output directory, it is impossible to avoid shared mutation if every rustdoc process is writing to the same CCI
437
+
*Every rustdoc process outputs the CCI to the same doc root by default
438
438
* It is difficult to extract the items in a diverse set of rendered HTML files. This is anticipating of the CCI to include HTML files that, for example, statically include type+trait implementations directly
439
439
* Reading exclusively from `doc.parts` is simpler than the existing `serde_json` dependency for extracting the blobs, as opposed to handwritten CCI-type specific parsing (current)
440
440
* With this proposal, there will be duplicate logic to read from both `doc.parts` files and rendered CCI.
@@ -455,11 +455,11 @@ There is a render option, `no_emit_shared`, which is used to conditionally gener
455
455
456
456
This option is not configurable from the command line, and appears to be enabled unless rustdoc is run in its example scraping mode.
457
457
458
-
We could make it configurable from the command line, unconditionally generate `doc.parts/`, and use it to gate the merging of CCI.
458
+
We could make it configurable from the command line, unconditionally generate `doc.parts`, and use it to gate the merging of CCI.
459
459
460
-
We could also make it configurable from the command line, and use it to gate the generation of `doc.parts/` and the generation of all of the shared files.
460
+
We could also make it configurable from the command line, and use it to gate the generation of `doc.parts` and the generation of all of the shared files.
461
461
462
-
We could also leave it as-is: always false unless we're scraping examples, and gate the generation of `doc.parts/` and the generation of all of the shared files.
462
+
We could also leave it as-is: always false unless we're scraping examples, and gate the generation of `doc.parts` and the generation of all of the shared files.
0 commit comments