Survey of Python package build environments

[sethmlarson]

Many Python packages are built using hosted build systems like GitHub Actions, Travis, and AppVeyor. Some build systems use containers, like cibuildwheel and multi-build. Survey the following information:

• Which operating systems are used?
• Which package managers are used?
• Do these package ecosystems have a Package URL (PURL)?
• Do these package ecosystems publish OSV records?
• Do these package ecosystems support querying from a filepath to a package?

Build environments to survey:

• multibuild
• cibuildwheel
• quay.io/pypa images

[sethmlarson]

Multibuild uses Travis for Linux and macOS and AppVeyor for Windows. Linux uses the quay.io/pypa manylinux images. Has a list of "libraries" that get built from source.

[sethmlarson]

The quay.io/pypa manylinux/musllinux images all use CentOS/AlmaLinux/Alpine operating systems, so the package managers are either yum, dnf or apk.

Package URLs:

• CentOS: pkg:rpm/centos/<package>@<version>
• AlmaLinux: pkg:rpm/almalinux/<package>@<version>
• Alpine: pkg:apk/alpine/<package>@<version>

Finding which package provides a file:

• yum: yum whatprovides <file>
• apk: apk info --who-provides <file>
• dnf: dnf provides <file>

[pitrou]

For PyArrow we use the following setups when building binary wheels:

• for manylinux, a Docker image based on the quay.io images, which pulls C++ dependencies using vcpkg; the build itself uses CMake + setuptools
• for macOS, a GHA setup; as above, vcpkg is used for C++ dependencies and the build uses CMake + setuptools
• for Windows, a Docker image; and, similarly, vcpkg for C++ dependencies, then CMake + setuptools

cc @raulcd @h-vetinari