Skip to content

Add QEMU multi-architecture support for PyInstaller builds#318

Merged
Ljzd-PRO merged 5 commits intodevelfrom
copilot/fix-fd828318-17b3-4a2c-9562-75614edcc3c1
Aug 18, 2025
Merged

Add QEMU multi-architecture support for PyInstaller builds#318
Ljzd-PRO merged 5 commits intodevelfrom
copilot/fix-fd828318-17b3-4a2c-9562-75614edcc3c1

Conversation

Copy link
Contributor

Copilot AI commented Aug 17, 2025

This PR implements QEMU-based cross-compilation support in the GitHub Actions workflow to enable PyInstaller builds across multiple architectures, addressing the need for broader platform support.

Changes Made

Enhanced Build Matrix

Extended the create-executable job matrix to support 6 different architecture combinations:

  • Windows: x86 (32-bit) and x64 (64-bit)
  • Linux: x64 (native) and arm64 (QEMU)
  • macOS: arm64 (Apple Silicon) and x64 (Intel)

QEMU Integration

Added QEMU emulation using docker/setup-qemu-action@v3 for Linux ARM builds:

- name: Set up QEMU
  if: matrix.qemu == true
  uses: docker/setup-qemu-action@v3
  with:
    platforms: ${{ matrix.qemu-arch }}

Dual Build Strategy

Implemented conditional build logic:

  • Native builds: Use existing PyInstaller action for optimal performance on Windows, macOS, and Linux x64
  • QEMU builds: Use Docker containers with python:3.13-slim-bullseye for ARM architectures

Updated Python Setup Action

Enhanced .github/actions/setup-python/action.yml to accept an architecture parameter, enabling Windows x86 builds alongside the default x64.

Improved Artifact Naming

Updated artifact names to include architecture information: ktoolbox-executable-${{ runner.os }}-${{ matrix.arch }}

Technical Details

The QEMU builds run inside Docker containers to ensure proper cross-compilation:

docker run --rm --platform linux/${{ matrix.qemu-arch }} \
  -v ${{ github.workspace }}:/workspace \
  -w /workspace \
  python:3.13-slim-bullseye \
  bash -c "
    apt-get update && apt-get install -y git binutils &&
    pip install poetry &&
    poetry install --with pyinstaller --extras urwid --extras uvloop &&
    poetry run pyinstaller ktoolbox.spec --distpath dist &&
    cp README.md README_zh-CN.md example.env .env LICENSE dist/
  "

This approach maintains consistency with the existing build process while extending support to ARM architectures that cannot be natively built on GitHub's x64 runners. The binutils package provides the objdump utility required by PyInstaller for Linux builds. Poetry is installed directly via pip to avoid dependency issues with pipx in the Debian Bullseye repositories.

Benefits

  1. Comprehensive Architecture Support: Users can now download pre-built executables for their specific platform
  2. Cross-Platform Compatibility: ARM devices (ARM servers) now have native executable support
  3. Windows 32-bit Support: Addresses legacy Windows systems that require x86 binaries
  4. Future-Proof Design: Easy to extend for additional architectures as needed

The implementation maintains backward compatibility while significantly expanding the project's reach across different hardware platforms.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: Ljzd-PRO <63289359+Ljzd-PRO@users.noreply.github.com>
Copilot AI changed the title [WIP] 给.github/workflows/python-package.yml增加使用qemu的action,在Linux的多种常见架构环境以及Windows的x86和amd64下使用pyinstaller构建 Add QEMU multi-architecture support for PyInstaller builds Aug 17, 2025
Copilot AI requested a review from Ljzd-PRO August 17, 2025 07:32
@Ljzd-PRO
Copy link
Owner

@copilot

Run # For QEMU builds, we need to build inside an emulated container
  # For QEMU builds, we need to build inside an emulated container
  docker run --rm --platform linux/arm \
    -v /home/runner/work/KToolBox/KToolBox:/workspace \
    -w /workspace \
    python:3.13-slim-bullseye \
    bash -c "
      apt-get update && apt-get install -y pipx git &&
      pipx install poetry &&
      export PATH=/root/.local/bin:\$PATH &&
      poetry install --with pyinstaller --extras urwid --extras uvloop &&
      poetry run pyinstaller ktoolbox.spec --distpath dist &&
      cp README.md README_zh-CN.md example.env .env LICENSE dist/
    "
  shell: /usr/bin/bash -e {0}
Unable to find image 'python:3.13-slim-bullseye' locally
3.13-slim-bullseye: Pulling from library/python
205b9d33caf0: Pulling fs layer
09d9815684a2: Pulling fs layer
432446270b36: Pulling fs layer
4aa2b3cbd8c3: Pulling fs layer
4aa2b3cbd8c3: Waiting
205b9d33caf0: Verifying Checksum
205b9d33caf0: Download complete
09d9815684a2: Verifying Checksum
09d9815684a2: Download complete
4aa2b3cbd8c3: Verifying Checksum
4aa2b3cbd8c3: Download complete
432446270b36: Verifying Checksum
432446270b36: Download complete
205b9d33caf0: Pull complete
09d9815684a2: Pull complete
432446270b36: Pull complete
4aa2b3cbd8c3: Pull complete
Digest: sha256:e98b521460ee75bca92175c16247bdf7275637a8faaeb2bcfa19d879ae5c4b9a
Status: Downloaded newer image for python:3.13-slim-bullseye
Get:1 http://deb.debian.org/debian bullseye InRelease [75.1 kB]
Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [27.2 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.0 kB]
Get:4 http://deb.debian.org/debian bullseye/main armhf Packages [7841 kB]
Get:5 http://deb.debian.org/debian-security bullseye-security/main armhf Packages [383 kB]
Get:6 http://deb.debian.org/debian bullseye-updates/main armhf Packages [16.3 kB]
Fetched 8387 kB in 7s (1224 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package pipx
Error: Process completed with exit code 100.

Co-authored-by: Ljzd-PRO <63289359+Ljzd-PRO@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Aug 17, 2025

@copilot

Run # For QEMU builds, we need to build inside an emulated container

...

Fixed the pipx installation error in the QEMU builds. The issue was that pipx is not available in the Debian Bullseye repositories for ARM architectures. I've updated the build script to use pip install poetry directly instead of trying to install and use pipx, which resolves the dependency issue. Commit: ff20780

@Ljzd-PRO
Copy link
Owner

@copilot 去除armv7,Linux只需要arm64和amd64

Co-authored-by: Ljzd-PRO <63289359+Ljzd-PRO@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Aug 17, 2025

@copilot 去除armv7,Linux只需要arm64和amd64

Removed armv7 architecture support from the Linux build matrix. The workflow now only builds for Linux x64 (amd64) and arm64 architectures as requested. Commit: a2b5209

@Ljzd-PRO Ljzd-PRO marked this pull request as ready for review August 17, 2025 08:42
@Ljzd-PRO
Copy link
Owner

@copilot

Installing the current project: ktoolbox (v0.19.2)
ERROR: On Linux, objdump is required. It is typically provided by the 'binutils' package installable via your Linux distribution's package manager.
Error: Process completed with exit code 1.

Co-authored-by: Ljzd-PRO <63289359+Ljzd-PRO@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Aug 17, 2025

@copilot

Installing the current project: ktoolbox (v0.19.2)

...

Fixed the PyInstaller objdump error by adding the binutils package to the QEMU build dependencies. The issue was that PyInstaller requires objdump which is provided by the binutils package, but it wasn't installed in the Docker container. Commit: 4f2d190

Copilot AI requested a review from Ljzd-PRO August 17, 2025 11:04
@Ljzd-PRO Ljzd-PRO merged commit e417949 into devel Aug 18, 2025
15 of 33 checks passed
@Ljzd-PRO Ljzd-PRO deleted the copilot/fix-fd828318-17b3-4a2c-9562-75614edcc3c1 branch August 18, 2025 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants