pypdfium2 does not need an additional system package, is generally faster than poppler, and used by one of your dependencies already for this very purpose (doctr).
Disclaimer: I'm the author of pypdfium2 and found this repository through GH dependents network since pypdfium2 is listed in your poetry.lock file.