How to make `-sys` crates deterministic

[hofbi]

Problem

We realized that all -sys crates such as zstd-sys or lz4-sys show non-deterministic behavior. This is likely caused by the build.rs scripts non using a proper CC toolchain but whatever the script does.

Our Solution

We identified a pattern to fix this non deterministic behavior by doing

# MODULE.bazel
crate.annotation(
    # Make zstd-sys build deterministically.
    additive_build_file = "//:zstd-sys.BUILD.bazel",
    crate = "zstd-sys",
    gen_build_script = "off",    # don't use the build.rs script to compile the C/C++ code
    repositories = ["crates"],
    deps = [":zstd"],                  # rely on your cc_library which is compiled using your CC toolchain
)

and zstd-sys.BUILD.bazel adding a cc_libary that brings in all the CC sources and headers, and is using your CC toolchian.

Next Steps

Since we (and probably everyone else) have to do this for every -sys dependency, we might want to find a better way of solving this. Maybe even through rules_rust or at least document this so that users are aware of this.

Relevant Work

• build -sys packages with rules_foreign_cc targtes #2759
• Build scripts fail on *-sys packages where the native library already supports bazel #3465
• cargo_build_script is non determnistic when compiling cc_libraries #3481