overlap text highlighting

[sadaqabdo]

When I try to highlight specific text in a PDF document, I have encountered issues with overlapping highlights and inconsistent contrast levels. The problem occurs when highlighting multiple instances of the same text or when adjacent texts are highlighted.

*Example to reproduce:
pdf:

import fitz
from fitz import Annot

ref = "Attention Is All You Need\nAshish Vaswani∗\nGoogle Brain\nBrain\n[email protected]\nNoam Shazeer∗\nGoogle Brain\n[email protected]\nNiki Parmar∗\nGoogle Research\n[email protected]\nJakob Uszkoreit∗\nGoogle Research\n[email protected]\nLlion Jones∗\nGoogle Research\n[email protected]\nAidan N. Gomez∗ †\nUniversity of Toronto\n[email protected]\nŁukasz Kaiser∗\nGoogle Brain\n[email protected]\nIllia Polosukhin∗ ‡\n[email protected]\nAbstract\nThe dominant sequence transduction models are based on complex recurrent or\nconvolutional neural networks that include an encoder and a decoder."

pdf = fitz.open("transformers.pdf")

ref_lines = ref.split("\n")
pages_to_ref = {0}

highlight_rects = []

for page in pdf:
    if page.number in pages_to_ref:
        blocks = page.get_textpage().extractBLOCKS()
        for block in blocks:
            block_texts = block[4].split("\n")
            for block_text in block_texts:
                if block_text in ref_lines:
                    for ref_line in ref_lines:
                        rls = page.search_for(ref_line, quads=False)
                        if rls is not None:
                            for rl in rls:
                                highlight_rects.append(
                                    {
                                        "pn": page.number, 
                                        "rect": rl,
                                    }
                                )

marked_area = set()
for highlight_rect in highlight_rects:
    for page in pdf:
        if page.number == highlight_rect["pn"]:
            if highlight_rect["rect"] not in marked_area:
                annot: Annot = page.add_highlight_annot(highlight_rect["rect"])
                annot.set_colors({"stroke":  (191/255, 255/255, 191/255)})
                annot.update()
                marked_area.add(highlight_rect["rect"])


pdf.save("transformers-highlighted.pdf", garbage=4, deflate=True, clean=True)

it results this, for word Brain:

any solution on how a word can only be highlighted one time?

[JorjMcKie]

Well, the word Brain occurs 2 times in that ref: "Google Brain" and "Brain" - so no wonder this happens. How about "Google" only?
Highlight annots in addition use blendmode "Multiply", therefore mutiple overlaps are darker than single ones.
Another thing you might want to try is not to create single annotations for every word hit, but to instead pass all the rectangles as a list to page.add_highlight_annot(). This will create one annot for the full list of rectangles.
All this also depends on your selected color (vis-a-vis the blendmode): If you use color green (0,1,0), you will see no problem.