New Project: pypdfium2

[mara004]

Please describe the project

pypdfium2 is a Python library for high-level PDF operations (e.g. rendering, text extraction), based on Google's PDFium.

The project is relatively young, but has high download count according to pepy.tech statistics (currently 7m/month). It is used for PDF ingestion by some popular AI software (e.g. langchain, docling, dify).

We endeavor to support as many platforms as possible with pre-built wheel packages.

URL for the project
https://github.com/pypdfium2-team/pypdfium2
https://pypi.org/project/pypdfium2/
https://pypdfium2.readthedocs.io/en/stable/

Describe current CI/CD setup
GitHub Actions (?)

Describe the primary use case for the Github Action Runner
Building release binary wheels for ppc64le and s390x (manylinux and musllinux) through cibuildwheel.

Paste a link to the existing actions workflow file(s) or directory
https://github.com/pypdfium2-team/pypdfium2/tree/main/.github/workflows/cibw.yaml
Maybe also https://github.com/pypdfium2-team/pypdfium2/tree/main/.github/workflows/sbuild_native.yaml once setup-python supports ppc64le and s390x, cf. actions/setup-python#1154.

How often do you plan on executing the runner?
On release, and for testing as-needed.

What is the primary programming language for the project?
Python, with C/C++ binary extension (ctypes-based).

Please select desired hardware

• Power 9 (ppc64le)
• IBM Z / LinuxONE (s390x)

Account names of the GitHub repo admins that will need access to setting up the runner
@mara004

Further Notes

For most platforms, pypdfium2 just repacks bblanchon's cross-compiled pdfium-binaries, but ppc64le/s390x are not handled in Google's toolchain yet.
A ppc64le downstream figured out that it theoretically can be done, but needs patches and self-built sysroot + libclang_rt builtins.

As this would require upstream changes at pdfium-binaries/pdfium, it is not clear if/when that approach could be taken, whereas building in a native runner with cibuildwheel is something we could do today from the pypdfium2 side.

Theoretically, we might be able to use emulation, but it is wastefully slow, e.g. 1h30 emulated vs 4-6min native. In case we need to build tests, or vendor more dependencies (e.g. ICU for smaller wheel sizes, or the libc++ for ABI safety), emulation becomes even less feasible.

Disclaimer

I'm a Python developer, not a compiled language expert, so we depend on a good basis from upstream.

pdfium's Readme acknowledges the possibility of endianness bugs (may affect s390x).
Also, the build strategy we are using (native tools & libraries) is not documented or officially supported upstream. However, the toolchain-based build they are suggesting is not portable.

[mtarsel]

Apologies for the delayed response here. We are ready to onboard this project now.

Please install the GitHub App from the following link and select your repository:
🔗 Power Z GHA Runner App

For more details on the onboarding process, please refer to our documentation:
📄 Onboarding Guide

Please let me know in this issue once you have installed the github app on your repo. For security reasons, I will double check it is the correct repo and then approve the repo to use our images. You will not be able to use our images until I approve the repo.

After the repo has been approved, you will be able to use the ubuntu-24.04-ppc64le and ubuntu-24.04-s390x images in the workflow file and all good to go :) The onboarding doc linked above will be helpful.

Please note: forks will not be able to use this service since those repos will not be in our allow list.

If there is any questions or concerns, we can discuss those here.

[mara004]

No problem, and thank you very much for the approval :) I'll follow the onboarding process soon.

As for questions or concerns, I'm not an expert on the matter, but #20 sounds indeed a bit worrying, in particular the "Administrative: Repository creation, deletion, settings, teams, and collaborators" permission. This must not get in the wrong hands. Also "Actions: workflows, workflow runs and artifacts" may be a loophole if workflows aren't vetted against untrusted inputs.
I understand there are internal difficulties, but it would be great if permissions could be tightened in the future, e.g. according to the model suggested by @tstellar in #20 (comment) ff.
However, as IBM is a reputable company, and many reputable projects are being onboarded here, I might trust and go ahead with the present permissions. But I can't deny this feels a bit concerning.

[mara004]

As for pypdfium2, I know our current cibuildwheel approach isn't ideal, but at least it is a quick and realsitic way of making wheels for Power and Z.
In particular, I'm not too happy about the idea of pulling in potentially outdated system libraries from a container, and would prefer a fully vendored build. This should be theoretically possible and is a future goal, but it's not something I could do today.

[mara004]

Hmm, as this potentially affects security, I figured I'd like to take into account a second opinion before moving on with this, after all.

@adam-huganir @mkroutikov Given that you are technically co-maintainers of pypdfium2 (though not currently active committers), may I ask for your POV on this matter?
Also (hoping he doesn't mind) I've tried to reach out to a workflow security expert on #20 (comment).

@mtarsel Sorry for the reluctance, I only read about #20 after I had submitted this proposal.
I still would like to support Power and Z, but at the same time I ought to avoid making a hasty move that might negatively impact the project's safety.

[adam-huganir]

Hmm, as this potentially affects security, I figured I'd like to take into account a second opinion before moving on with this, after all.

@mara004 i've definitely installed apps from less reputable places with more permissions in more sensitive situations, but that probably isn't the best argument. if the demand is there for these arches i'd say it's worth it since as you say it's not fun to emulate. worst case scenario seems both unlikely and manageable. theft/ip isn't an issue of course since there isn't much accessible that isn't already accessible publicly already. as for supply chain stuff, there is monitoring that could be set up to watch for unexpected changes anywhere, but with the app owner it would be below my threshold of paranoia for something like this.

tl;dr, seems reasonable to me given there is/will be demand for the builds and ibm being the app org. regular backups of github specific stuff like org stuff/issues/logs/anything else etc wouldn't be a bad idea in addition to external repo backups as well.