Auto merge of #137535 - Kobzol:split-metadata, r=petrochenkov

Introduce `-Zembed-metadata` to allow omitting full metadata from rlibs and dylibs

This is a continuation of rust-lang/rust#120855 (I was mentored by `@bjorn3` to move it forward). Most of the original code was written by bjorn3, I tried to clean it up a bit and add some documentation and tests.

This PR introduces a new unstable compiler flag called `-Zembed-metadata=[no|yes]`, with the default being `yes` (see rust-lang/rust#57076 for context). When set to `no`, rustc will only store a small metadata stub inside rlibs/dylibs instead of the full metadata, to keep their size smaller. It should be used in combination with `--emit=metadata`, so that the users of such a compiled library can still read the metadata from the corresponding `.rmeta` file. [This comment](rust-lang/rust#120855 (comment)) shows an example of binary/artifact size wins that can be achieved using this approach.

Contrary to rust-lang/rust#120855, this PR only introduces the new flag, along with a couple of run-make tests and documentation, but does not yet use it in bootstrap to actually compile rustc. I plan to do that as a follow-up step (along with integration in Cargo, which should ideally just always pass this flag to reduce the size of target directories).

Fixes rust-lang/rust#23366
Closes rust-lang/rust#29511
Fixes rust-lang/rust#57076

Another attempt of rust-lang/rust#93945 and rust-lang/rust#120855.

r? `@petrochenkov`

Author: bors

crates/proc-macro-srv/src/dylib/version.rs

@@ -110,15 +110,15 @@ pub fn read_version(obj: &object::File<'_>) -> io::Result<String> {
         ));
     }
     let version = u32::from_be_bytes([dot_rustc[4], dot_rustc[5], dot_rustc[6], dot_rustc[7]]);
-    // Last supported version is:
+    // Last breaking version change is:
     // https://github.com/rust-lang/rust/commit/b94cfefc860715fb2adf72a6955423d384c69318
     let (mut metadata_portion, bytes_before_version) = match version {
         8 => {
             let len_bytes = &dot_rustc[8..12];
             let data_len = u32::from_be_bytes(len_bytes.try_into().unwrap()) as usize;
             (&dot_rustc[12..data_len + 12], 13)
         }
-        9 => {
+        9 | 10 => {
             let len_bytes = &dot_rustc[8..16];
             let data_len = u64::from_le_bytes(len_bytes.try_into().unwrap()) as usize;
             (&dot_rustc[16..data_len + 12], 17)