Skip to content

Conversation

mara004
Copy link
Member

@mara004 mara004 commented Aug 7, 2025

@mara004
Copy link
Member Author

mara004 commented Aug 8, 2025

@wojiushixiaobai FYI, as you seem to be using this patch downstream already:
I think I'm going to drop cibuildwheel-conditional default vendoring of ICU and ask clients to pass --vendor icu explicitly, because I'm not sure how to handle overrides without clutter (--vendor without arguments is ambiguous).

@mara004 mara004 force-pushed the main branch 2 times, most recently from a14e153 to 5d4f312 Compare August 10, 2025 08:51
@mara004
Copy link
Member Author

mara004 commented Aug 13, 2025

I'm starting to notice I might not be too happy with cibuildwheel...

The abstraction layer may be convenient, but it is also restrictive.
Moreover, it seems that cibuildwheel is unable to do cross-compilation, and too tied to emulation / native runners (on linux, anyway). Might be more flexible to set up the containers and run auditwheel repair manually on our side?

Unfortunately, I don't know docker yet, and am unable to install it locally (outdated system).

@mara004
Copy link
Member Author

mara004 commented Aug 13, 2025

Ok, I made some progress to get docker (and cibuildwheel) running locally on Fedora after all:

sudo dnf in moby-engine  # this provides the docker command
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -aG docker $USER
# then reboot, e.g. via `systemctl reboot`

Then make a fresh checkout of pypdfium2 (in particular, to avoid an existing toolchained checkout), otherwise you might get a halt on the "Copy project into container" step:

git clone https://github.com/pypdfium2-team/pypdfium2.git pypdfium2_fresh
cd pypdfium2_fresh/
git checkout cibuildwheel  # current dev branch

Now we should be ready to run cibuildwheel:

export CIBW_BUILD="cp38-manylinux_x86_64"  # p.ex.
export CIBW_BEFORE_BUILD="python setupsrc/pypdfium2_setup/build_native.py --vendor icu"
export CIBW_ENVIRONMENT="PDFIUM_PLATFORM=sourcebuild"
export CIBW_BEFORE_ALL_LINUX="dnf -y install gn ninja-build freetype-devel glib2-devel lcms2-devel libjpeg-devel libpng-devel libtiff-devel openjpeg2-devel zlib-devel"
cibuildwheel . --output-dir wheelhouse

We can test the resulting wheel in a virtual env:

python3 -m venv testenv
./testenv/bin/pip install wheelhouse/pypdfium2-*-py3-none-manylinux_2_28_x86_64.whl
./testenv/bin/pypdfium2 -v
./testenv/bin/pip install pillow
mkdir out
./testenv/bin/pypdfium2 render ~/Downloads/$SOME_DOC.pdf --scale 3 -o out/

@mara004 mara004 force-pushed the cibuildwheel branch 2 times, most recently from 393bf55 to e168ef8 Compare August 21, 2025 08:36
@mara004 mara004 marked this pull request as ready for review August 21, 2025 09:45
@mara004
Copy link
Member Author

mara004 commented Aug 21, 2025

I believe this is ready for merge.

@wojiushixiaobai If you have anything to add, feel free to suggest changes and I'll try to incorporate them in main or a follow-up PR.

@mara004 mara004 merged commit 47da6da into main Aug 21, 2025
@mara004 mara004 deleted the cibuildwheel branch August 21, 2025 09:49
@mara004
Copy link
Member Author

mara004 commented Aug 21, 2025

@wojiushixiaobai Also, I just field IBM/actionspz#37 as you suggested on #339 (comment).

@@ -19,8 +19,9 @@ jobs:
fail-fast: false
matrix:
# On GH actions, macOS <=13 is Intel, whereas macOS >=14 will be ARM64
# Google's toolchain doesn't seem to run on Linux arm64 natively. The toolchain-free build (or cross-compilation from x86_64) should work, though.
os: ['ubuntu-latest', 'macos-13', 'macos-latest', 'windows-latest'] # 'ubuntu-24.04-arm'
# Google's toolchain doesn't seem to run on Linux/Windows arm64 natively. The toolchain-free build (or cross-compilation from x86_64) should work, though.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

xxx: native sourcebuild doesn't work on Windows yet - need to cross compile from amd64 in that case.
probably best to split this up in two separate comments

builder = dict(native=build_native, toolchained=build_toolchained).get(sub_target)
if builder:
build_params = vars(builder.parse_args(build_params))
build_params.update(dict(build_ver=pdfium_ver))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: do this only if build_ver evaluates to True / is non-None to avoid always shadowing --version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants