Rollup merge of rust-lang#148169 - fmease:rustdoc-bad-intra-bad-preprocess, r=lolbinarycat

Fix bad intra-doc-link preprocessing

How did rust-lang#147981 happen?

1. We don't parse intra-doc links as Rust paths or qpaths. Instead they follow a very lenient bespoke grammar. We completely ignore Markdown links if they contain characters that don't match `/[a-zA-Z0-9_:<>, !*&;]/` (we don't even emit lint *broken-intra-doc-links* for these).
2. PR [rust-lang#132748](rust-lang#132748) made rustdoc intepret more Markdown links as potential intra-doc links. Namely, if the link is surrounded by backticks (and some other conditions apply) then it doesn't matter if the (partially processed) link contains bad characters as defined above (cc `ignore_urllike && should_ignore_link(path_str)`).
3. However, rustdoc's `preprocess_link` must be kept in sync with a simplified counterpart in rustc. More specifically, whenever rustdoc's preprocessor returns a successful result then rustc's must yield the same result. Otherwise, rustc doesn't resolve the necessary links for rustdoc.
4. This uncovered a "dormant bug" / "mistake" in rustc's `preprocess_link`. Namely, when presented with a link like `struct@Type@suffix`, it didn't cut off the disambiguator if present (here: `struct@`). Instead it `rsplit('`@')`` which is incorrect if the "path" contains ``@`` itself (yielding `suffix` instead of `Type@suffix` here). Prior to PR [rust-lang#132748](rust-lang#132748), a link like ``[`struct@Type@suffix`]`` was not considered a potential intra-doc link / worth querying rustc for. Now it is due to the backticks.
5. Finally, since rustc didn't record a resolution for `Type@suffix` (it only recorded `suffix` (to be `Res::Err`)), we triggered an assertion we have in place to catch cases like this.

Fixes rust-lang#147981.

I didn't and still don't have the time to investigate if rust-lang#132748 led to more rustc/rustdoc mismatches (after all, the PR made rustdoc's `preprocess_link` return `Some(Ok(_))` in more cases). I've at least added another much needed "warning banner" & made the existing one more flashy.

While this fixes a stable-to-beta regression, I don't think it's worth beta backporting, esp. since it's only P-medium and since the final 1.91 release steps commence today / the next days, so it would only be stressful to get it in on time. However, feel free to nominate.

<sub>(I've written such a verbose PR description since I tend to reread my old PR descriptions in the far future to fully freshen my memories when I have to work again in this area)</sub>

r? `@lolbinarycat`

Author: matthiaskrgr

compiler/rustc_resolve/src/rustdoc.rs

@@ -393,13 +393,15 @@ pub fn has_primitive_or_keyword_or_attribute_docs(attrs: &[impl AttributeExt]) -
 }
 
 /// Simplified version of the corresponding function in rustdoc.
-/// If the rustdoc version returns a successful result, this function must return the same result.
-/// Otherwise this function may return anything.
 fn preprocess_link(link: &str) -> Box<str> {
+    // IMPORTANT: To be kept in sync with the corresponding function in rustdoc.
+    // Namely, whenever the rustdoc function returns a successful result for a given input,
+    // this function *MUST* return a link that's equal to `PreprocessingInfo.path_str`!
+
     let link = link.replace('`', "");
     let link = link.split('#').next().unwrap();
     let link = link.trim();
-    let link = link.rsplit('@').next().unwrap();
+    let link = link.split_once('@').map_or(link, |(_, rhs)| rhs);
     let link = link.trim_suffix("()");
     let link = link.trim_suffix("{}");
     let link = link.trim_suffix("[]");

src/librustdoc/passes/collect_intra_doc_links.rs

@@ -901,14 +901,18 @@ pub(crate) struct PreprocessedMarkdownLink(
 
 /// Returns:
 /// - `None` if the link should be ignored.
-/// - `Some(Err)` if the link should emit an error
-/// - `Some(Ok)` if the link is valid
+/// - `Some(Err(_))` if the link should emit an error
+/// - `Some(Ok(_))` if the link is valid
 ///
 /// `link_buffer` is needed for lifetime reasons; it will always be overwritten and the contents ignored.
 fn preprocess_link(
     ori_link: &MarkdownLink,
     dox: &str,
 ) -> Option<Result<PreprocessingInfo, PreprocessingError>> {
+    // IMPORTANT: To be kept in sync with the corresponding function in `rustc_resolve::rustdoc`.
+    // Namely, whenever this function returns a successful result for a given input,
+    // the rustc counterpart *MUST* return a link that's equal to `PreprocessingInfo.path_str`!
+
     // certain link kinds cannot have their path be urls,
     // so they should not be ignored, no matter how much they look like urls.
     // e.g. [https://example.com/] is not a link to example.com.

tests/rustdoc-ui/intra-doc/empty-associated-items.rs

@@ -0,0 +1,17 @@
+// This test ensures that (syntactically) malformed paths will not crash rustdoc.
+#![deny(rustdoc::broken_intra_doc_links)]
+
+// This is a regression test for <https://github.com/rust-lang/rust/issues/140026>.
+//! [`Type::`]
+//~^ ERROR
+
+// This is a regression test for <https://github.com/rust-lang/rust/issues/147981>.
+//! [`struct@Type@field`]
+//~^ ERROR
+
+//! [Type&content]
+//~^ ERROR
+//! [`Type::field%extra`]
+//~^ ERROR
+
+pub struct Type { pub field: () }

tests/rustdoc-ui/intra-doc/malformed-paths.rs

@@ -0,0 +1,36 @@
+error: unresolved link to `Type::`
+  --> $DIR/malformed-paths.rs:5:7
+   |
+LL | //! [`Type::`]
+   |       ^^^^^^ the struct `Type` has no field or associated item named ``
+   |
+note: the lint level is defined here
+  --> $DIR/malformed-paths.rs:2:9
+   |
+LL | #![deny(rustdoc::broken_intra_doc_links)]
+   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+error: unresolved link to `Type@field`
+  --> $DIR/malformed-paths.rs:9:7
+   |
+LL | //! [`struct@Type@field`]
+   |       ^^^^^^^^^^^^^^^^^ no item named `Type@field` in scope
+   |
+   = help: to escape `[` and `]` characters, add '\' before them like `\[` or `\]`
+
+error: unresolved link to `Type&content`
+  --> $DIR/malformed-paths.rs:12:6
+   |
+LL | //! [Type&content]
+   |      ^^^^^^^^^^^^ no item named `Type&content` in scope
+   |
+   = help: to escape `[` and `]` characters, add '\' before them like `\[` or `\]`
+
+error: unresolved link to `Type::field%extra`
+  --> $DIR/malformed-paths.rs:14:7
+   |
+LL | //! [`Type::field%extra`]
+   |       ^^^^^^^^^^^^^^^^^ the struct `Type` has no field or associated item named `field%extra`
+
+error: aborting due to 4 previous errors
+