Skip to content

Conversation

plobsing
Copy link
Contributor

@plobsing plobsing commented Jul 31, 2025

Cross builds of bundled applications would be really nice to be able to do.
Often, we're bundling for the purposes of distributing an artifact, and in many cases
the ultimate execution targets for the distributed artifact do not necessarily match
the build machine.

The PyPex action itself can support cross-builds in a fairly straightforward way:
use the cfg="exec" Python to run the PEX bundling code, rather than the
cfg="target" Python. The former is what this ruleset's PEX wrapper is already
running as, so all the must be done is to not request an alternative interpreter
when calling into the PEX library. Unfortunately, that's precisely what the tool
had been doing.

For instance, prior to this change, building an x86-64 Linux PEX binary on Macos
resulted in failures with diagnostics of the form:

pex.executor.Executor.ExecutionError: OSError(8, 'Exec format error') while trying to execute `['/private/var/tmp/_bazel_peter/0576637e8cc9ee3ad4c778fd98f4cba7/external/python_toolchain_x86_64-unknown-linux-gnu/bin/python3.9', [...]

The way rules_py is using PEX, changing the interpreter should not have other
meaningful effects. Perhaps if we were
allowing PEX to resolve requirements freely, changing the interpreter would change
the resolution results. However, that's non-hermetic, and the rules_py PEX wrapper
already explicitly controls what dependencies are used and where they come from.

Unfortunately, we are only able to support cross-builds targetting Linux at the current time
(and perhaps MacOS in release rulesets).
Other accidental complexity blocks targetting of other platforms.
See #625 for details.
However, even limited to just Linux as a cross target, the feature is valuable as Linux is very popular in
certain contexts.


Changes are visible to end-users: yes

  • Searched for relevant documentation and updated as needed: yes
  • Breaking change (forces users to change their own code or config): no
  • Suggested release notes appear below: no

Test plan

  • Covered by existing test cases
  • New test cases added

plobsing added 2 commits July 30, 2025 23:05
The target interpreter might not work all the time. In a cross-build,
the target interpreter will be for the target OS and the target
architecture, which will not match the host OS and/or architecture.
Copy link

aspect-workflows bot commented Jul 31, 2025

Test

1 test target passed

Targets
//py/tests/py-pex-binary:cross_builds [k8-fastbuild]                 30ms

Total test execution time was 30ms. 41 tests (97.6%) were fully cached saving 1m 10s.

@plobsing
Copy link
Contributor Author

Test failure is a true positive. It detected a case where cross builds still don't work: cross-compiling to Mac from Linux. The reason it does this is because the Mac Rust toolchains have target_compatible_with restricting to only Mac OS. However, that's actually not what we're trying to do with these Rust toolchains; we're using Rust to compile py/tools/venv_bin which is itself a toolchain (that is, the Rust binary we're building will run on the build host, a Linux machine). I believe there's a transition missing somewhere, although I'm not sure where yet. Help definitely appreciated.

@arrdem arrdem self-assigned this Aug 14, 2025
@plobsing plobsing changed the title feat(pex): support cross builds of py_pex_binary feat(pex): support cross builds of py_pex_binary targetting Linux Aug 16, 2025
@plobsing
Copy link
Contributor Author

Test failure is a true positive. It detected a case where cross builds still don't work: cross-compiling to Mac from Linux. The reason it does this is because the Mac Rust toolchains have target_compatible_with restricting to only Mac OS. However, that's actually not what we're trying to do with these Rust toolchains; we're using Rust to compile py/tools/venv_bin which is itself a toolchain (that is, the Rust binary we're building will run on the build host, a Linux machine). I believe there's a transition missing somewhere, although I'm not sure where yet. Help definitely appreciated.

FYI, after investigating, I've determined that the best path forward is to downscope this PR to just
cross builds that target Linux. Which is still useful for building PEX bundles for things like AWS Lambda.

There is additional complexity in expanding cross-build support to targets beyond Linux.
I think doing that would be neat, and #625 describes in greater
detail what is needed there.
Support for non-Linux cross build targets still needs the changes from this PR, its just there are also other,
orthogonal, additional things that must be fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants