Modernization Phase 2: Replace shell-based file discovery with structured WIT bindgen outputs

[avrabe]

Problem

The WIT bindgen rule in wit/wit_bindgen.bzl uses shell-based file discovery patterns that break cross-platform compatibility and violate Bazel's hermetic build principles.

Current Implementation Issues

File: wit/wit_bindgen.bzl (Lines 101-120, 139-150)

# Shell-based file discovery
ctx.actions.run_shell(
    command = """
        # Find generated .rs file and copy to expected location
        for f in {out_dir}/*.rs; do
            if [ -f "$f" ]; then
                cp "$f" {out_file}
                exit 0
            fi
        done
        echo "Error: No .rs file generated by wit-bindgen" >&2
        exit 1
    """.format(out_dir=out_dir.path, out_file=out_file.path),
    # ...
)

# Complex dependency structure creation with shell commands
ctx.actions.run_shell(
    command = """
        # Create dependency structure
        mkdir -p {deps_dir}
        
        # Create symlinks to WIT library directory
        for f in {wit_lib_dir}/*; do ln -sf "$(realpath "$f")" {deps_dir}/; done
        
        # Run wit-bindgen from deps directory
        cd {deps_dir}
        WIT_FILE=$(basename {wit_file})
        "$(pwd)/../{wit_bindgen}" {lang_args} "$WIT_FILE"
    """,
    # ...
)

Impact

• Windows incompatibility: Commands like ln -sf, realpath, cd don't work reliably on Windows
• Unpredictable outputs: Shell file discovery makes builds non-deterministic
• Hermetic violations: Relies on system shell, filesystem commands, and PATH
• Complex debugging: Shell script failures are harder to debug than structured Bazel actions
• Security concerns: Complex shell scripts increase attack surface

Solution

Replace shell-based file discovery with structured wit-bindgen CLI usage and predictable output paths.

Implementation Plan

1. Use wit-bindgen's --out-file parameter for predictable outputs:

   def _generate_rust_bindings(ctx, wit_bindgen, bindgen_args, inputs, out_file):
       # Use structured approach instead of shell file discovery
       ctx.actions.run(
           executable = wit_bindgen,
           arguments = bindgen_args + ["--out-file", out_file.path],
           inputs = inputs,
           outputs = [out_file],  # Predictable output, no shell discovery needed
           mnemonic = "WitBindgen"
       )

2. Replace dependency structure creation with Bazel-native operations:

   # Instead of shell symlinks, use ctx.actions.symlink
   def _create_wit_workspace(ctx, wit_library_dir, workspace_dir):
       ctx.actions.run(
           executable = "//tools:wit_workspace_setup",  # Custom tool
           arguments = [wit_library_dir.path, workspace_dir.path],
           inputs = [wit_library_dir],
           outputs = [workspace_dir],
           mnemonic = "WitWorkspaceSetup"
       )

3. Alternative: Use wit-bindgen's newer CLI options that support workspace-relative paths:

   # Research wit-bindgen --help to find workspace/dependency options
   # that eliminate need for complex directory setup

Files to Update

• wit/wit_bindgen.bzl - Replace shell file discovery and dependency setup
• Consider creating //tools:wit_workspace_setup helper tool if needed
• Update any rules that depend on WIT bindgen outputs

Research Required

• Investigate wit-bindgen CLI options for predictable output paths
• Determine if wit-bindgen supports workspace-relative dependency resolution
• Test with various WIT dependency structures

Testing

• Test WIT binding generation with and without external dependencies
• Verify cross-platform compatibility (Windows, macOS, Linux)
• Check that all language targets (Rust, C, etc.) work correctly
• Test complex WIT dependency graphs

Acceptance Criteria

• No shell commands in WIT bindgen actions
• Predictable, deterministic output file paths
• All WIT dependency scenarios supported
• Cross-platform compatibility verified
• Performance equal or better than current implementation
• All existing WIT bindgen functionality preserved

Priority

HIGH - Critical for Windows compatibility and hermetic builds.

Dependencies

This issue is independent but complements Issue #40 (File Operations Component).

Research Notes

The solution may require updating to newer wit-bindgen versions or creating a custom workspace setup tool.