Enhancements tracking issue

[mara004]

Users / non-maintainers, please do not comment on this issue! Open your own issue or discussions ticket instead.

---

This is a meta-issue to track miscellaneous tasks / possible enhancements that may not merit an own issue, but should be kept in mind (regard it as the maintainer's notes to self).

Runtime

• Add a pypdfium2_config submodule or similar to let the caller to customize the init phase, e.g. custom font paths. See also feat: Add custom font initialization for PDFium #380 (comment)

Setup / CI

• If possible, avoid our setup code running twice on pip install. cibuildwheel FAQ recommends running codegen in build_py class, but not sure what impact that may have on control flow and metadata consistency in our case. (Prio: High)

• Properly determine the Linux glibc tags instead of guessing and hardcoding them. Consider doing it as a post-processing step with auditwheel in craft.py. See Confusion around minimum required glibc version (linux) #203. (Prio: High)

• On Windows / in refbindings, expose windows-only members, such as FPDF_RenderPage (to windows device context). (Activity in forks shows this may be desired by some embedders.)

• In the packaging workflow, switch from a temporary branch to patchfile transfer? (Prio: Low)

• cibuildwheel: figure out why we are getting an awkward 2_27/2_28 dual-tag which makes no sense. Might need to file an issue upstream. See build_native testing on the linux/s390x #368 (reply in thread).

• Create a workflow that aggregates and publishes the intersection of the cibuildwheel / sourcebuild workflows? See Sourcebuild workflow refactoring #377 (comment) ff.

• Rename potentially misleading stable branch to docs. See Rename misleading stable branch to docs #342 (Prio: Low)

• Test platforms not provided by GH actions via emulation. See Test platforms not provided by GH actions via emulation #339 (Prio: Low)

• Add testing workflow for PDFIUM_PLATFORM=system installation, if feasible. (Prio: Low)

• autorelease: add comment field for the pin, instead of having it in base.py which needs manual taking care of. (Prio: Low)

• setupsrc/ chores:

  • Use proper enums instead of string classes.
  • Add ability to access abstracted OS/CPU separately through sub-attributes for PlatNames.
  • Add a proper mapping from uname to Google CPU names.
  • Align our own platform names with either uname or Google?
  • Try to unbloat and defragment setup code. Avoid introducing new bloat.
  • Give pdfium-binaries targets a prefix, instead of just the platform name?

Runtime sources

• New Features

  • Matrix render helper
  • CLI
    • input mode selection
    • ability to render multiple files at once

• Enhancements

  • Use Py_IncRef() / Py_DecRef() instead of wonky reference holder objects. (Also affects Readme Raw API examples.) (Prio: Medium)
  • Think about mutex sharing, and plugging in a mutex into the autoclose machinery. See also Crashed in a multi-threaded environment datalab-to/marker#225 (comment) ff. (Prio: Medium)

[mara004]

update.py: Consider passing --format=json to gh at verify. Run it only on one artifact, and check the others by hashes in the JSON output. Assert against more fields.

done in 29dd119 (at least the first part, assertions TBD)

[mara004]

Avoid potentially bloating sys.path in imports boilerplate. Might need to check if ... not in sys.path first? (Prio: Medium)

done in daf5ec9, a4caaf5, 5a85f3e but I'm not entirely sure it's good what we ended up with

[mara004]

Hmm, I realized I'm not happy about how gh attestation verify is integrated.
I don't like the bloated workaround code, and its use in workflows is particularly unfortunate as we often hit limits for GH unauthenticated API e.g. in test_setup.yaml so verification isn't actually active on a subset of runners.
I would prefer an "always authenticated" use in our workflows, but without exposing the token to a larger code surface.
It could be done by keeping the pdfium-binaries tgz's in a directory and running verification in a separate script and step.
Once that's done, we can think about how much workaround code we want to remove or not remove.

[mara004]

Also logging the build script enhancement task here, as outlined on IBM/actionspz#37 (comment)

[mara004]

More possible enhancements:

• build_native.py: Add ability to vendor all dependency libraries. In particular, when building with non-system clang (e.g. https://github.com/mayeut/static-clang-images), it might be desirable to use the vendored libc++. Vendoring the other libraries reduces build dependencies and increases the chance that this might work on an unhandled host without much need for manual interaction.
• Add android targets to sbuild.yaml. It shouldn't be too difficult, as build_toolchained.py already supports it.

[mara004]

Some (much?) of the above has now been implemented/addressed.

Extending minitest.py to at least render something would also be great. We can't run the test suite on all platforms because Pillow/numpy aren't easily available everywhere. Though, in the future, using container's system python & system packages may be an option. That means we'd definitely have to maintain a low python requirement.

Alternatively, making test suite dependencies optional, and adding a dedicated dependency-less rendering test case, might work.

[mara004]

Our Readme needs an overhaul again, specifically the sourcebuild / cibuildwheel sections.
Update: done

[mara004]

Also, regarding native sourcebuild, the cibuildwheel legacy libs patch can be dropped if vendored libs are used.

[mara004]

Oh, and the loongarch64 build ought to be included in the attestation. This was an oversight.
Update: Done