Skip to content

Commit a072524

Browse files
jaimergpahesford
andcommitted
add some more notes from #1841
Co-authored-by: ahesford <[email protected]>
1 parent 8647982 commit a072524

File tree

1 file changed

+45
-9
lines changed

1 file changed

+45
-9
lines changed

sphinx/src/maintainer/knowledge_base.rst

Lines changed: 45 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -308,9 +308,29 @@ A package that needs all five compilers would define
308308
Cross-compilation
309309
-----------------
310310

311-
For some other architectures (like ARM), packages can be built natively on that architecture or they can be cross-compiled.
312-
In other words, built on a different common architecture (like x86_64) while still targeting the original architecture (ARM).
313-
This helps one leverage more abundant CI resources in the build architecture (x86_64).
311+
conda-forge defaults to native builds of packages for x86_64 on Linux, macOS and Windows, because
312+
that's the architecture powering the default CI runners. Other architectures are supported too,
313+
but they are not guaranteed to have native builds. In those platforms where we can't provide native
314+
CI runners, we can still resort to either cross-compilation or emulation.
315+
316+
Cross-compiling means building a package for a different architecture than the one the build process
317+
is running on. Given how abundant x86_64 runners are, most common cross-compilation setups will target
318+
non-x86_64 architectures from x86_64 runners.
319+
320+
Cross-compilation terminology usually distinguishes between two types of machine:
321+
322+
- Build: The machine running the building process.
323+
- Host: The machine we are building packages for.
324+
325+
.. note::
326+
327+
Some cross-compilation documentation might also distinguish between a third type of machine, the
328+
target machine. You can read more about it in `this Stack Overflow question
329+
<https://stackoverflow.com/questions/47010422/cross-compilation-terminologies-build-host-and-target>`__.
330+
For the purposes of conda-forge, we'll consider the target machine to be the same as the host.
331+
332+
How to enable cross-compilation
333+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
314334

315335
Cross-compilation settings depend on the ``build_platform`` and ``target_platform`` conda-build
316336
variables:
@@ -358,6 +378,17 @@ This is all supported by two main conda-build features introduced in version 3:
358378
- The ``compiler()`` Jinja function and underlying `conventions for the compiler packages
359379
<https://docs.conda.io/projects/conda-build/en/latest/resources/compiler-tools.html>`__.
360380

381+
Placing requirements in build or host
382+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
383+
384+
The rule of the thumb is:
385+
386+
- If it needs to run during the build, it goes in ``build``.
387+
- If it needs to be available on the target host, it goes in ``host``.
388+
- If both conditions are true, it belongs in both.
389+
390+
However, there are some exceptions to this rule; most notably Python cross-compilation (see below).
391+
361392
Cross-compilation examples
362393
^^^^^^^^^^^^^^^^^^^^^^^^^^
363394

@@ -384,7 +415,7 @@ cross-compiling:
384415
cp $BUILD_PREFIX/share/gnuconfig/config.* .
385416
386417
# Skip ``make check`` when cross-compiling
387-
if [[ "${CONDA_BUILD_CROSS_COMPILATION}" != "1" ]]; then
418+
if [[ "${CONDA_BUILD_CROSS_COMPILATION:-}" != "1" || "${CROSSCOMPILING_EMULATOR:-}" != "" ]]; then
388419
make check
389420
fi
390421
@@ -456,11 +487,15 @@ but merely to provide a starting point with some guidelines. Please look at `oth
456487
Details about cross-compiled Python packages
457488
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
458489

459-
Cross-compiling Python packages is a bit more involved than other packages. This is because Python
460-
doesn't have official support for cross-compilation and a series of workarounds need to be applied
461-
to make it work. See `PEP720 <https://peps.python.org/pep-0720/>`__ for more details.
490+
Cross-compiling Python packages is a bit more involved than other packages. The main pain point is
491+
that we need an executable Python interpreter (i.e. ``python`` in ``build``) that knows how to
492+
provide accurate information about the target platform. Since this is not officially supported, a
493+
series of workarounds are required to make it work. Refer to `PEP720
494+
<https://peps.python.org/pep-0720/>`__ or `the discussion in this issue
495+
<https://github.com/conda-forge/conda-forge.github.io/issues/1841>`__ for more information.
462496

463-
On conda-forge, there are two extra metadata bits that are needed in ``meta.yaml``:
497+
In practical terms, for conda-forge, this results into two extra metadata bits that are needed in
498+
``meta.yaml``:
464499

465500
- Adding ``cross-python_{{ target_platform }}`` in ``build`` requirements, provided by the
466501
`cross-python-feedstock <https://github.com/conda-forge/cross-python-feedstock>`__. This is a
@@ -488,7 +523,8 @@ following changes before the builds scripts run:
488523

489524
All in all, this results in a setup where ``conda-build`` can run a ``$BUILD_PREFIX``-architecture
490525
``python`` interpreter that can see the packages in ``$PREFIX`` (with the compiled bits provided by
491-
their corresponding counterparts in ``$BUILD_PREFIX``) and mimic that target architecture.
526+
their corresponding counterparts in ``$BUILD_PREFIX``) and sufficiently mimic that target
527+
architecture.
492528

493529
Rust Nightly
494530
------------

0 commit comments

Comments
 (0)