[API View] Pin nightly for rustdoc JSON output (Azure#2356)

* Pin nightly-2025-01-12

* NIGHTLY_TOOLCHAIN_FOR_APIVIEW variable in rust.yml

* revert rust.yml

* Use rust-toolchain.toml

* Remove NIGHTLY_TOOLCHAIN_FOR_APIVIEW variable

* Update eng/pipelines/templates/variables/rust.yml

* error message to run from the root

---------

Co-authored-by: Heath Stewart <[email protected]>

Author: HarshaNalluru

eng/tools/generate_api_report/Cargo.toml

@@ -6,6 +6,10 @@ license = "MIT"
 repository = "https://github.com/azure/azure-sdk-for-rust"
 rust-version = "1.80"
 
+[build-dependencies]
+serde = { version = "1.0", features = ["derive"] }
+toml = "0.8.20"
+
 [dependencies]
 rustdoc-types = "0.33.0"
 serde_json = "1.0"

eng/tools/generate_api_report/README.md

@@ -7,28 +7,38 @@ It uses the following command `cargo +nightly rustdoc -Z unstable-options --outp
 
 ## Version Compatibility
 
-When updating the `rustdoc-types` in Cargo.toml, ensure that the `rust-api-parser` in the tools repository is also updated along with this project.
+The project depends on specific version relationships between several components:
 
-1. Check the `FORMAT_VERSION` value in the new version.
-2. Verify that it is compatible with your rustdoc JSON output version (generated by `cargo +nightly rustdoc ...`).
-3. If they do not match, you may need to use a different version of `rustdoc-types`.
+1. **rustdoc FORMAT_VERSION**: The rustdoc JSON output has a specific `FORMAT_VERSION` (currently 37). Different nightly versions of Rust may produce different format versions.
 
-Current `FORMAT_VERSION`: 37 (as of rustdoc-types 0.33.0)
+2. **rustdoc-types crate**: The version of this dependency in Cargo.toml (currently 0.33.0) must be compatible with the JSON format version produced by the selected nightly toolchain.
+
+3. **rust-api-parser**: This project in the azure-sdk-for-tools repository consumes the JSON files produced by this project and depends on the **rustdoc-types crate**. When updating the above components, ensure that the rust-api-parser tool is also updated to maintain compatibility.
+
+### Version Update Process
+
+When updating the nightly toolchain or the rustdoc-types crate, follow these steps:
+
+- First check the `FORMAT_VERSION` in a sample JSON output from the new nightly
+- Update the rustdoc-types crate version to match
+- Update `rust-toolchain.toml` with the desired nightly toolchain version and run `rustup install`.
+- Update the rust-api-parser project in the azure-sdk-for-tools repository to ensure compatibility with the new JSON format
+- Test the complete workflow to ensure all tools in the chain remain compatible
 
 ## Usage
 
 To run the tool, navigate to the root of the `azure-sdk-for-rust` repository and use the following command:
 
 ```sh
-    cargo run --manifest-path eng/tools/generate_api_report/Cargo.toml -- --package package_name
+cargo run --manifest-path eng/tools/generate_api_report/Cargo.toml -- --package package_name
 ```
 
 Generates `package_name.rust.json` in the `sdk/service_folder/package_name/review` directory.
 
 For example, to generate the report for a package named `azure_core`, run:
 
 ```bash
-    cargo run --manifest-path eng/tools/generate_api_report/Cargo.toml -- --package azure_core
+cargo run --manifest-path eng/tools/generate_api_report/Cargo.toml -- --package azure_core
 ```
 
 ## Functionality

eng/tools/generate_api_report/build.rs

@@ -0,0 +1,28 @@
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT License.
+
+use serde::Deserialize;
+use std::{fs, path::Path};
+
+fn main() {
+    let toolchain_file = Path::new(env!("CARGO_MANIFEST_DIR")).join("rust-toolchain.toml");
+    let toolchain_content =
+        fs::read_to_string(toolchain_file).expect("read rust-toolchain.toml from crate root");
+    let manifest: Manifest =
+        toml::from_str(&toolchain_content).expect("deserialize rust-toolchain.toml");
+    println!(
+        "cargo::rustc-env=TOOLCHAIN_CHANNEL={}",
+        manifest.toolchain.channel
+    );
+    println!("cargo::rerun-if-changed=rust-toolchain.toml");
+}
+
+#[derive(Debug, Deserialize)]
+struct Manifest {
+    toolchain: Toolchain,
+}
+
+#[derive(Debug, Deserialize)]
+struct Toolchain {
+    channel: String,
+}

eng/tools/generate_api_report/rust-toolchain.toml

@@ -0,0 +1,7 @@
+[toolchain]
+channel = "nightly-2025-01-12"
+components = [
+  "cargo",
+  "rust-docs",
+  "rust-std",
+]

eng/tools/generate_api_report/src/main.rs

@@ -1,12 +1,23 @@
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT License.
+
 use rustdoc_types::Crate;
 use std::env;
 use std::error::Error;
+use std::ffi::OsStr;
 use std::fs::File;
 use std::io::prelude::*;
 use std::path::Path;
 use std::process::Command;
 
 fn main() -> Result<(), Box<dyn Error>> {
+    // Verify we're running from the repository root
+    if !Path::new("eng/tools/generate_api_report").exists() {
+        return Err(
+            "This tool must be run from the root of the azure-sdk-for-rust repository. Use: cargo run --manifest-path eng/tools/generate_api_report/Cargo.toml -- --package {package_name}".into(),
+        );
+    }
+
     // Get the package name from command-line arguments
     let args: Vec<String> = env::args().collect();
     if args.len() != 3 || args[1] != "--package" {
@@ -18,17 +29,28 @@ fn main() -> Result<(), Box<dyn Error>> {
     let path = Path::new(&path_str);
 
     // Call cargo +nightly rustdoc to generate the JSON file
-    let output = Command::new("cargo")
-        .arg("+nightly")
+    let channel = env!("TOOLCHAIN_CHANNEL");
+    let mut command = Command::new("cargo");
+    command
+        .arg(format!("+{channel}"))
         .arg("rustdoc")
         .arg("-Z")
         .arg("unstable-options")
         .arg("--output-format")
         .arg("json")
         .arg("--package")
         .arg(package_name)
-        .arg("--all-features")
-        .output()?;
+        .arg("--all-features");
+    println!(
+        "Running: {} {}",
+        command.get_program().to_string_lossy(),
+        command
+            .get_args()
+            .collect::<Vec<&OsStr>>()
+            .join(OsStr::new(" "))
+            .to_string_lossy(),
+    );
+    let output = command.output()?;
 
     if !output.status.success() {
         eprintln!(
@@ -42,6 +64,8 @@ fn main() -> Result<(), Box<dyn Error>> {
     let mut contents = String::new();
     file.read_to_string(&mut contents)?;
 
+    println!("Processing rustdoc output for package: {}", package_name);
+
     let mut root: Crate = serde_json::from_str(&contents)?;
 
     // Remove items