multiprocessing errors

[aihiangng]

Hi all, I am trying to use multiprocessing to extract text from all the pages. However, im getting the following error when trying to open each page to process:

mupdf: expected object number mupdf: expected object number mupdf: expected object number mupdf: expected object number mupdf: expected object number mupdf: zlib error: incorrect header check mupdf: expected object number mupdf: corrupt object stream (138556 0 R) mupdf: zlib error: unknown compression method

Any help is appreciated!! Thank you

[JorjMcKie]

Need more information! The PDF is probably corrupt.
Only happens in multiprocessing mode?

[aihiangng]

Hi! It works when im extracting it page by page with a for loop. However when I use multiprocessing to open and process each page thats when I get an error. I get the error when its trying to open the pages with this:

page=doc[page_number]

It throws an error in the above line of code.

[JorjMcKie]

There is an example script in the docu. You should use this as a start.
Th most important things to watch in this area:

1. PyMuPDF does not support Python multithreading, only multiprocessing.
2. Every process needs to open its own Document object(s) - cannot be shared across processes, although it may be the same physical file of course. So you need some logic to hand a page range to each procress.
3. Data spanning multiple page ranges must be carefully handled by Python's multiprocessing features. E.g. to create any totals or whatever.
4. It is not possible to update the same file in parallel by different processes.