Bazel: rules_multirun Python dependency prevents local `bazel run` with remote execution

[mfreeman451]

Problem

When using remote execution (--config=remote) from macOS to build Linux Docker images, bazel run //docker/images:push_all fails with:

OSError: [Errno 8] Exec format error: '.../rules_python++python+python_3_11_x86_64-unknown-linux-gnu/bin/python3'

The same issue affects individual oci_push targets (they embed Linux jq binary).

Root Cause

rules_multirun is implemented in Python. When using remote execution:

1. Bazel builds targets for the execution platform (Linux x86_64)
2. Host-transitioned targets (like rules_multirun) embed Linux Python in runfiles
3. bazel run executes locally on macOS
4. Linux binaries cannot execute on macOS → "Exec format error"

This is a known limitation of Bazel's platform model when host ≠ execution platform.

Upstream References

• Issue: Consider dropping Python dependency keith/rules_multirun#43 (open since 2023)
• Go Port PR: Go port and windows support keith/rules_multirun#57 (stalled since June 2024)
• Aspect blog on prebuilt binaries: https://blog.aspect.build/releasing-bazel-rulesets-rust

The maintainer is "generally supportive" of switching to Go but PR #57 has been waiting for review. The PR also has a bug where go_sdk extension isn't properly guarded as dev_dependency.

Current Workarounds

1. Push from CI only - GitHub Actions runs on Linux where host = execution platform
2. scripts/push_all_images.sh - Wrapper script that explains the limitation
3. Build works fine - make build / bazel build //... works with remote execution

What Still Works

Action	Works on macOS?	Notes
make build	✅	Builds all images via RBE
bazel build //docker/images:...	✅	All images built correctly
bazel run //docker/images:push_all	❌	Python exec format error
bazel run from Linux CI	✅	Host matches execution platform

Potential Solutions

1. Wait for upstream - Monitor PR improve getChecker() method #57 for progress
2. Fork and fix PR improve getChecker() method #57 - Fix the dev_dependency bug, maintain our own fork
3. Replace rules_multirun - Simple shell script that calls each push target
4. Contribute upstream - Help get PR improve getChecker() method #57 merged

Files with Documentation

• MODULE.bazel - Comment explaining the issue
• .bazelrc - Note about bazel run limitation with remote execution
• scripts/push_all_images.sh - Workaround script for CI
• Makefile - Updated push_all target

Labels

• bazel
• toolchain
• wontfix (for now)