Layout detection model embedded

[gabriela0neagu]

Hello,

Is there any possibility to access and further train the existing layout detection model?

Thank you

[JorjMcKie]

Can you elaborate on your question? Are you referring to the heuristics producing the blocks / lines / spans hierarchy?

[gabriela0neagu]

Yes, I assumed that the library has a NN model behind for layout detection. I am interested in the logic for image extraction, since there are some cases when the library does not detect them(when they are on the entire document page). I was hoping there is a model that maybe I could possibly further train foe better results.

[JorjMcKie]

The code you are referring to is inside the base library MuPDF, which is written in C - and not based on any NN or AI technologies.

If it were, its market leadership in terms of performance and small footprint never could have been achieved. Please keep in mind that MuPDF runs on lots of platforms, including smartphones, where it often enough is the only PDF viewer that can process certain documents at all or with acceptable performance.

Its superior technical architecture also allows processing multiple disparate document types beyond PDF (XPS, e-books, and more) with no change for an application layer on top of MuPDF. This was the leading design principle.
At the same time, an application has access to all detail present in a document and may thus use its own logic to come to desired results.

As per PDF text extraction, please also keep in mind, that this document type allows placing single characters on arbitrary positions on a page. So for a page showing text with N characters, there are up to N! different ways to encode the page with exactly the same appearance.
And this is not just a theoretical possibility: examples exist which exploit this fact to impede extraction of the text (notwithstanding that PDF has many more ways to protect content).
So the hierarchy block -> line -> span -> character is - and must ever remain - a guess, albeit covering a large portion of real world cases.

You may not be aware that there is a way to extract text in layout-preserving manner. This is an example for what application code can achieve using low-level information provided by the base library.

As per your comment on failing image recognition: I don't believe this is true. So please do provide an example.

[gabriela0neagu]

Thank you for the clarifications regarding the logic behind the library. I sent you an email with the pdf samples on which I was unable to get the expected results. I am waiting forward to your response.