Selecting a sentence, part of which is highlighted or annotated

[dummifiedme]

Wanted to discuss, if there is a way we can have sentences extracted from a pdf, part of which is highlighted or annotated.

It might be off topic, but would like to know if someone has ideas on how to get thats sort of a thing done.
I tried exploring 'nltk' library. But integrating it with the PyMuPDF way of things, seems to be difficult?

If someone has done some work in that direction, please share :)

[JorjMcKie]

Highlighting takes places in the form of annotations.
And how text can be extracted, that is contained in annotation rectangles has already been discussed elsewhere. It basically works by extracting text from a series of rectangles defined by the annotation rectangles.
Of course there is no way to know if the thus extracted text makes up a complete sentence - if that was the direction of your question.
Suchthings are out of PyMuPDF scope and more a matter of semantics or some AI.

The opposite, namely how to locate and then highlight text that spans across several lines, is being discussed in some detail in here.

[dummifiedme]

Okay. Got it.
I was thinking on the lines of appraching it like:

• look for highlighted text.
• Select the first word from the highlights
• Check for the previous word in the wordlist
• check till t is a . or word.
• Join them all and similiarly for the other end.

For simple use case, can something like this possible or I am just ignoring all the things there is?

And yes, I know this is beyond the scope PyMuPDF :) I hope you don't mind.

Regarding the opposite, yes, I had gone through that link. :D

[JorjMcKie]

I hope you don't mind.

Of course not.
And yes: backward and forward searching until start / end of sentence is found, may work / can word / should work.
But only if a number of conditions are true or, vice versa, some complications can be excluded:

• Page layout only has one text column
• extracted text pieces of the highlights can neatly be matched with extracted text, via getText("words"), or the spans of getText("dict")
• partly overlapping the previous: what happens, if text fonts / fontsizes change within a line, or text pieces only roughly have the same baseline, but actually do differ by 1 or 2 pixels, ... In such cases some cleaning / simplification has to happen first so that these portions are forced on the same line and in the proper sequence

[dummifiedme]

Yes, there would be complications, I agree.
I thought the word list which getText(“words”) captures is like a chain of words. It doesn’t really have any format right? It’s essentially just a list of words? So, if it is so, from what I could understand, I can traverse through it word by word and find the ends of sentences.
But yes, as you said, there might be complications due to changing fonts or inability of the function to capture the words properly. Again, that inability will anyway effect if I am capturing the highlight. I mean, that “inability” would be at both the places, in that case we can’t rely on extractions any way (ie in case of a “bad” PDF).

What is the difference between different arguments being passed in getText()? I only used “words” till now. What does “dict” do? And what are the other arguments that could be used?
I will try to read over in the documentation in my weekends maybe 👍

Also, does getText(“words”) capture a line of words without considering the columns? I tried it with a PDF with boxed contents, there it kind of stopped at the edge of the box and started to capture next line. It captured the boxed text later on. I thought it was working on something like “Blocks”, I don’t know if its really the case.

[JorjMcKie]

Also, does getText(“words”) capture a line of words without considering the columns?

Yes, exactly. But each word item also contains block number, line number in the block, word number in the line. So you would presumably sort the words according to their page position (using the 4 leading bbox floats). If things are not too awkward, then pages with multiple text columns would be expected to have different block numbers for different columns. So block number would be a good candidate as the leading sort key. Of course, a text column may still consist of more than one block.

[dummifiedme]

Got it. I still have to learn about bbox. But I get what you are saying. To have a more robust and fail safe code, I should have a sound sorting. Currently, I just loop around the annotations and capture whatever comes up. So far so good on my documents. AS you had suggested somewhere in the discussion, I will have to eventually sort the things according to their positions. I will at some point :)

[JorjMcKie]

I know that @zeusrb is doing stuff in that direction. Maybe he wants to jump in and share his position or findings.

[zeusrb]

BTW, I' keeping the good chunks to retrofit my AI engine, so next round will do better search and document categorization ... ;-)

[dummifiedme]

It will take some days if not more to wrap my head around it. I hope I can take this up long term as a hobby along with my studies.

Thanks to PyMuPDF, my workflow became a bit smooth.

The ability to get the sentences from incomplete highlights would be something I would like to have sometime in future.
PS It's really interesting to learn, but the Exams!😐

[JorjMcKie]

What subjects are you studying?

[dummifiedme]

I am preparing for administrative services. So it involves basically everything :)
If I have to list the subjects, it would be somewhat around:

• History (Ancient India, Medieval India, Modern India, After Independence, World History)
• Political Science and governance (Mostly about Indian Polity)
• Economics (again revolves around Indian economy and other economic policies that might in turn affect India, eg oil)
• Geography and Disaster Management ( physical, human and Indian geography)
• International Relations (Mostly around indian relations and policies that effect India)
• Art and Culture (Architecture, Dances,, etc)
• Environment Studies
• Defence and Security
• Ethics and the related aptitude.
• And of course Science :p

Other than these, we need to be aware of current developments. And an optional subject which is completely personal choice. I chose Anthropology (despite being a mechanical design engineer 😉 )

So yeah, that’s about it.

[JorjMcKie]

Uff, not really can be called a sort list 🙄

[navjeetc]

I have a similar requirement to extract only highlighted text from multiple pages. It is not clear to me from above if that can be successfully done. If yes, a small snippet of code would be nice.

[JorjMcKie]

Depending on how "highlighting" has been done technically, this can be easy or complex.
The easy case:

• Iterate over the annotations of a page and down-select the highlight annotations.
• Take the annotation rectangle and extract the text covered by it.

for annot in page.annots(types=[fitz.PDF_ANNOT_HIGHLIGHT]):
    rect = annot.rect  # annotation rectangle
    rect += (-5, -5, 5, 5)  # enlarge it a bit to cope with imprecisions
    print(page.get_textbox(rect))  # the text

If this does not deliver as expected, the highlighting may have been done by vector graphics.
Please ping me if you need more support.

[navjeetc]

This is very helpful. Thanks.

[JorjMcKie]

my invitation stands...