OCI to sysroot utility

[kdowney-lot49]

I am most of the way in getting a way to convert lightweight images created with apko into sysroots for use in cross-compilation, which lets me create them on-the-fly using the same base image as the one I plan to use for deployment. Unfortunately the way sysroots fit into toolchains_llvm is a bit opaque to me. I created a genrule that emits a depset with the extracted files and it correctly creates the sysroot in the directory structure, but then what gets passed to the compiler is wrong.

What is the label below expected to point to? What I'm getting passed to clang is --sysroot=build-support/sysroot rather than --sysroot=build-support/sysroot/sysroot_amd64 as expected, so I am guessing the depset() being passed back with the extracted files is the wrong thing to provide.

llvm.sysroot(
    name = "llvm_toolchain",
    label = "@//build-support/sysroot:sysroot_amd64",
    targets = ["linux-x86_64"],
)

This is sysroot rule definition; all it does is run a Python script that takes the OCI image and unpacks it:

build-support/sysroot/defs.bzl

def _sysroot_impl(ctx):
    output_dir = ctx.actions.declare_directory(ctx.attr.name)
    
    src_files = ctx.files.src
    if not src_files:
        fail("No source files provided")
        
    # We use the first file to determine the layout directory path
    layout_path = src_files[0].path
    # If it's a file inside the layout, we'll handle dirname in the script.
    
    args = ctx.actions.args()
    args.add(ctx.file.tool.path)
    args.add(layout_path)
    args.add(output_dir.path)
    
    ctx.actions.run(
        outputs = [output_dir],
        inputs = src_files + [ctx.file.tool],
        executable = "python3",
        arguments = [args],
        use_default_shell_env = True,
        mnemonic = "ExtractSysroot",
        progress_message = "Extracting sysroot...",
    )
    
    # My guess is this is not what's expected as output, but then what is?
    return [DefaultInfo(files = depset([output_dir]))]

sysroot = rule(
    implementation = _sysroot_impl,
    attrs = {
        "src": attr.label(mandatory = True, allow_files = True),
        "tool": attr.label(mandatory = True, allow_single_file = True),
    },
)

build-support/sysroot/BUILD.bazel

genrule(
    name = "sysroot_archive_amd64",
    srcs = ["sysroot.yaml", "sysroot.lock.json"],
    outs = ["sysroot_amd64.tar"],
    tools = ["@rules_apko//apko:apko"],
    cmd = "$(location @rules_apko//apko:apko) build $(location sysroot.yaml) latest-amd64 $@ --arch amd64",
    tags = ["requires-network"],
)

sysroot(
    name = "sysroot_amd64",
    src = ":sysroot_archive_amd64",
    tool = ":extract_sysroot.py",
)

If I can get it working and cleaned up I'm happy to put it on GitHub, and then it might be helpful to add to the examples.

[kdowney-lot49]

OK, I haven't figured out a workaround yet, but I know why it's failing (and this behavior is documented, not a bug):

    label = Label(sysroot)
    sysroot_path = _pkg_path_from_label(label)

If sysroot is given a label instead of an absolute path, it's translated into the package path. The problem is that's not the path to the extracted sysroot, but the location of defs.bzl. It needs to be resolved into an absolute path to work.

[kdowney-lot49]

I figured this out -- I had to create a repository_rule instead, and furthermore that rule had to download its own apko binary. But once I had that working, I was able to create a wolfi-base sysroot and then compile both amd64 and arm64 binaries.