Question about PyMuPDF4LLM

[dkoterwa]

Hi,
I've wanted to ask if there is any difference in text processing when it comes to get_text("blocks") vs the way PyMuPDF4LLM processes text.
I know that for each rectangle with text you define a Markdown tag to set the level of the header, but is there anything more apart from that? I am not asking about tables/images processing, just the raw text.

[JorjMcKie]

PyMuPDF4LLM uses an advanced way of detecting text blocks on a page, compared to get_text("blocks"). This allows it detecting multi-column pages more reliably.

[dkoterwa]

Thank you for a quick reply. Could you please specify what does "an advanced way" mean? I can see that in PyMuPDF4LLM you are joining bboxes that overlap and make sure that bboxes won't be concatenated across columns. If we assume that the PDF document is single-column, then can we expect that extracted blocks and markdown elements will be very similar across these two methods?

[JorjMcKie]

Roughly true, yes.

But please be aware that in PDF we cannot rely on text being nicely ordered in a way that makes sense semantically. The "blocks" will always follow their physical enumeration sequence in the respective PDF source.
Even when sorted (sort=True parameter), single text piece may remain out of order because they may have been added / replaced by some editing mechanism.
Many / most of these pesky things are resolved in to_markdown() because text flow is re-synthesized from a longer detail level upwards. Which is part of the reason why this extraction type is significantly slower.

[dkoterwa]

Everything I've wanted to ask for is clarified now. Thank you for your help, I think that this discussion can be closed.

[JorjMcKie]

Thanks for the interesting post.
Discussions need not be closed. We usually even leave them open so they are visible for the user community.