Difference in bbox rectangle of text items

[qwertynik]

The rect returned for text by the page.get_bboxlog() is different (probably 'incorrect' too) in comparison to what is returned by bbox within span accessed via page.get_text("dict")["blocks"]:

Not sure of the reasons behind the discrepancy.

Rectangle drawn using rect from bboxlog

Rectangle drawn using rect from get_text() - looks correct.

Edit: Rectangle drawn using rect from get_texttrace() - further different.

Source PDF

Can you add clarity here? @JorjMcKie

[JorjMcKie]

You forgot to do ´page.clean_contents()` again.
After doing that everything looks fine.

[qwertynik]

Hmm, tried it. But it does not work. The output still looks the same. Sample high-level script.

import fitz

import util

pdfPath = "pdfs/RPHT9.pdf"
pdfPath = "pdfs/PHT35_OnlyShapes.pdf"
doc = fitz.open(pdfPath)
page = doc[0]
page.clean_contents()

bboxList = page.get_bboxlog()

textTrace = page.get_texttrace()

textSpans = util.get_text_spans_in_pdf(doc)

All 3 output are different. The output from texttrace and bboxlog continue to remain 'incorrect' and different too. The output from text_span fetch is also now appearing incorrectly. @JorjMcKie

[LeMoussel]

In which module is get_text_spans_in_pdf? What is the utilmodule?

[qwertynik]

@LeMoussel
util is a custom module.

[JorjMcKie]

here is how it looks like drawing the bboxlog rectangles for subtypes "stroke-parh" and "fill-path"
x.pdf
.

[JorjMcKie]

and here are the text trace results:

page.clean_contents()
spans=page.get_texttrace()
for s in spans:
    page.draw_rect(s["bbox"])

    
Point(220.17300415039062, 199.83132934570312)
Point(160.72500610351562, 262.7900390625)
Point(107.70500183105469, 146.6465606689453)
Point(111.16899871826172, 355.25250244140625)
doc.save("y.pdf")

y.pdf

[qwertynik]

@JorjMcKie

In this output, notice how the 2nd rect, exceeds the text's boundary in the PDF.

[JorjMcKie]

everything as it should be, so I don't know what you were doing, and you code does not really tell.

[qwertynik]

Attached a snippet only this time because for the image rendering internal utility methods are used. However, had observed similar behaviour in the output you shared.
Replied here

[JorjMcKie]

In this output, notice how the 2nd rect, exceeds the text's boundary in the PDF.

What do you mean? There is nothing wrong with it.

[qwertynik]

Assuming the difference in the output between both the approaches is *now clear.

[JorjMcKie]

That is no argument.

You did not read the documentation (closely enough): .get_texttrace() is a low-level technical method and has little or nothing to do with .get_text() - where MuPDF already did some preprocessing. Both have no code in common inside PyMuPDF.
.get_texttrace() does absolutely nothing but returning every single text character in exactly the sequence as it is encountered. New spans are created if and only if MuPDF opens a new span within a "fill-text". Which in turn is related to encountering a new PDF text block wrapped in commands "BT"/"ET". Not when characters have been written far away from each other or whatever.
Exactly as it happens in mutool trace.

[qwertynik]

You did not read the documentation (closely enough): .get_texttrace() is a low-level technical method and has
little or nothing to do with .get_text()

Yes, @JorjMcKie had read the first paragraph of documentation for .get_texttrace() and started using it. However, as you mentioned, and as also observed in a section documentation solely dedicated to this, realized that the output of get_texttrace does not process the text as get_text would do. On this page the documentation helped in understanding on what the get_text method uses under the hood.

To reiterate this is not seen as a bug in the library - but the idea is to understand the difference.

Ran mutool trace, and noticed two span tags, this explains why the bbox rect from get_texttrace drew two rectangles.

<fill_text colorspace="DeviceRGB" color=".639216 .639216 .639216" transform="1 0 0 -1 0 2016">
    <span font="Helvetica" wmode="0" bidi="0" trm="14 0 0 14">
        <g unicode="c" glyph="c" x="107.705" y="1858.4" adv=".5"/>
        <g unicode="o" glyph="o" x="114.705" y="1858.4" adv=".556"/>
        <g unicode="O" glyph="O" x="122.489009" y="1858.4" adv=".778"/>
        <g unicode="r" glyph="r" x="133.38101" y="1858.4" adv=".333"/>
        <g unicode="d" glyph="d" x="138.04302" y="1858.4" adv=".556"/>
        <g unicode="i" glyph="i" x="145.82701" y="1858.4" adv=".222"/>
        <g unicode="n" glyph="n" x="148.93502" y="1858.4" adv=".556"/>
        <g unicode="a" glyph="a" x="156.71901" y="1858.4" adv=".556"/>
        <g unicode="t" glyph="t" x="164.503" y="1858.4" adv=".278"/>
        <g unicode="e" glyph="e" x="168.395" y="1858.4" adv=".556"/>
        <g unicode="s" glyph="s" x="176.179" y="1858.4" adv=".5"/>
        <g unicode=" " glyph="space" x="183.179" y="1858.4" adv=".278"/>
        <g unicode="=" glyph="equal" x="187.071" y="1858.4" adv=".584"/>
        <g unicode=" " glyph="space" x="195.247" y="1858.4" adv=".278"/>
        <g unicode="(" glyph="parenleft" x="199.13899" y="1858.4" adv=".333"/>
        <g unicode="0" glyph="zero" x="203.801" y="1858.4" adv=".556"/>
        <g unicode="," glyph="comma" x="211.58499" y="1858.4" adv=".278"/>
        <g unicode=" " glyph="space" x="215.47699" y="1858.4" adv=".278"/>
        <g unicode="0" glyph="zero" x="219.36899" y="1858.4" adv=".556"/>
        <g unicode="," glyph="comma" x="227.15299" y="1858.4" adv=".278"/>
        <g unicode=" " glyph="space" x="231.04499" y="1858.4" adv=".278"/>
        <g unicode="9" glyph="nine" x="234.93698" y="1858.4" adv=".556"/>
        <g unicode="0" glyph="zero" x="242.72098" y="1858.4" adv=".556"/>
        <g unicode="0" glyph="zero" x="250.50498" y="1858.4" adv=".556"/>
        <g unicode="," glyph="comma" x="258.28898" y="1858.4" adv=".278"/>
        <g unicode=" " glyph="space" x="262.18098" y="1858.4" adv=".278"/>
        <g unicode="3" glyph="three" x="266.07298" y="1858.4" adv=".556"/>
        <g unicode="0" glyph="zero" x="273.85697" y="1858.4" adv=".556"/>
        <g unicode="0" glyph="zero" x="281.64097" y="1858.4" adv=".556"/>
        <g unicode=")" glyph="parenright" x="289.42497" y="1858.4" adv=".333"/>
        <g unicode=" " glyph="space" x="294.08695" y="1858.4" adv=".278"/>
        <g unicode="/" glyph="slash" x="297.97895" y="1858.4" adv=".278"/>
        <g unicode="/" glyph="slash" x="301.87095" y="1858.4" adv=".278"/>
        <g unicode="x" glyph="x" x="305.76295" y="1858.4" adv=".5"/>
        <g unicode="0" glyph="zero" x="312.76295" y="1858.4" adv=".556"/>
        <g unicode="," glyph="comma" x="320.54695" y="1858.4" adv=".278"/>
        <g unicode=" " glyph="space" x="324.43894" y="1858.4" adv=".278"/>
        <g unicode="y" glyph="y" x="328.33094" y="1858.4" adv=".5"/>
        <g unicode="0" glyph="zero" x="335.33094" y="1858.4" adv=".556"/>
        <g unicode="," glyph="comma" x="343.11494" y="1858.4" adv=".278"/>
        <g unicode=" " glyph="space" x="347.00694" y="1858.4" adv=".278"/>
        <g unicode="x" glyph="x" x="350.89894" y="1858.4" adv=".5"/>
        <g unicode="1" glyph="one" x="357.89894" y="1858.4" adv=".556"/>
        <g unicode="," glyph="comma" x="365.68293" y="1858.4" adv=".278"/>
        <g unicode=" " glyph="space" x="369.57493" y="1858.4" adv=".278"/>
        <g unicode="y" glyph="y" x="373.46693" y="1858.4" adv=".5"/>
        <g unicode="1" glyph="one" x="380.46693" y="1858.4" adv=".556"/>
    </span>
    <span font="Helvetica" wmode="0" bidi="0" trm="14 0 0 14">
        <g unicode="c" glyph="c" x="111.169" y="1649.7941" adv=".5"/>
        <g unicode="o" glyph="o" x="118.169" y="1649.7941" adv=".556"/>
        <g unicode="O" glyph="O" x="125.953" y="1649.7941" adv=".778"/>
        <g unicode="r" glyph="r" x="136.845" y="1649.7941" adv=".333"/>
        <g unicode="d" glyph="d" x="141.507" y="1649.7941" adv=".556"/>
        <g unicode="i" glyph="i" x="149.291" y="1649.7941" adv=".222"/>
        <g unicode="n" glyph="n" x="152.399" y="1649.7941" adv=".556"/>
        <g unicode="a" glyph="a" x="160.183" y="1649.7941" adv=".556"/>
        <g unicode="t" glyph="t" x="167.967" y="1649.7941" adv=".278"/>
        <g unicode="e" glyph="e" x="171.859" y="1649.7941" adv=".556"/>
        <g unicode="s" glyph="s" x="179.64299" y="1649.7941" adv=".5"/>
        <g unicode=" " glyph="space" x="186.64299" y="1649.7941" adv=".278"/>
        <g unicode="=" glyph="equal" x="190.53499" y="1649.7941" adv=".584"/>
        <g unicode=" " glyph="space" x="198.71099" y="1649.7941" adv=".278"/>
        <g unicode="(" glyph="parenleft" x="202.60298" y="1649.7941" adv=".333"/>
        <g unicode="0" glyph="zero" x="207.26499" y="1649.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="215.04898" y="1649.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="218.94098" y="1649.7941" adv=".278"/>
        <g unicode="2" glyph="two" x="222.83298" y="1649.7941" adv=".556"/>
        <g unicode="5" glyph="five" x="230.61698" y="1649.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="238.40097" y="1649.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="246.18497" y="1649.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="250.07697" y="1649.7941" adv=".278"/>
        <g unicode="9" glyph="nine" x="253.96897" y="1649.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="261.75297" y="1649.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="269.53697" y="1649.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="277.32096" y="1649.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="281.21296" y="1649.7941" adv=".278"/>
        <g unicode="5" glyph="five" x="285.10496" y="1649.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="292.88896" y="1649.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="300.67295" y="1649.7941" adv=".556"/>
        <g unicode=")" glyph="parenright" x="308.45695" y="1649.7941" adv=".333"/>
        <g unicode=" " glyph="space" x="313.11894" y="1649.7941" adv=".278"/>
        <g unicode="/" glyph="slash" x="317.01094" y="1649.7941" adv=".278"/>
        <g unicode="/" glyph="slash" x="320.90293" y="1649.7941" adv=".278"/>
        <g unicode="x" glyph="x" x="324.79493" y="1649.7941" adv=".5"/>
        <g unicode="0" glyph="zero" x="331.79493" y="1649.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="339.57893" y="1649.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="343.47093" y="1649.7941" adv=".278"/>
        <g unicode="y" glyph="y" x="347.3629" y="1649.7941" adv=".5"/>
        <g unicode="0" glyph="zero" x="354.3629" y="1649.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="362.1469" y="1649.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="366.0389" y="1649.7941" adv=".278"/>
        <g unicode="x" glyph="x" x="369.9309" y="1649.7941" adv=".5"/>
        <g unicode="1" glyph="one" x="376.9309" y="1649.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="384.7149" y="1649.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="388.6069" y="1649.7941" adv=".278"/>
        <g unicode="y" glyph="y" x="392.4989" y="1649.7941" adv=".5"/>
        <g unicode="1" glyph="one" x="399.4989" y="1649.7941" adv=".556"/>
        <g unicode="c" glyph="c" x="111.169" y="1457.7941" adv=".5"/>
        <g unicode="o" glyph="o" x="118.169" y="1457.7941" adv=".556"/>
        <g unicode="O" glyph="O" x="125.953" y="1457.7941" adv=".778"/>
        <g unicode="r" glyph="r" x="136.845" y="1457.7941" adv=".333"/>
        <g unicode="d" glyph="d" x="141.507" y="1457.7941" adv=".556"/>
        <g unicode="i" glyph="i" x="149.291" y="1457.7941" adv=".222"/>
        <g unicode="n" glyph="n" x="152.399" y="1457.7941" adv=".556"/>
        <g unicode="a" glyph="a" x="160.183" y="1457.7941" adv=".556"/>
        <g unicode="t" glyph="t" x="167.967" y="1457.7941" adv=".278"/>
        <g unicode="e" glyph="e" x="171.859" y="1457.7941" adv=".556"/>
        <g unicode="s" glyph="s" x="179.64299" y="1457.7941" adv=".5"/>
        <g unicode=" " glyph="space" x="186.64299" y="1457.7941" adv=".278"/>
        <g unicode="=" glyph="equal" x="190.53499" y="1457.7941" adv=".584"/>
        <g unicode=" " glyph="space" x="198.71099" y="1457.7941" adv=".278"/>
        <g unicode="(" glyph="parenleft" x="202.60298" y="1457.7941" adv=".333"/>
        <g unicode="0" glyph="zero" x="207.26499" y="1457.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="215.04898" y="1457.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="218.94098" y="1457.7941" adv=".278"/>
        <g unicode="5" glyph="five" x="222.83298" y="1457.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="230.61698" y="1457.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="238.40097" y="1457.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="246.18497" y="1457.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="250.07697" y="1457.7941" adv=".278"/>
        <g unicode="9" glyph="nine" x="253.96897" y="1457.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="261.75297" y="1457.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="269.53697" y="1457.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="277.32096" y="1457.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="281.21296" y="1457.7941" adv=".278"/>
        <g unicode="8" glyph="eight" x="285.10496" y="1457.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="292.88896" y="1457.7941" adv=".556"/>
        <g unicode="0" glyph="zero" x="300.67295" y="1457.7941" adv=".556"/>
        <g unicode=")" glyph="parenright" x="308.45695" y="1457.7941" adv=".333"/>
        <g unicode=" " glyph="space" x="313.11894" y="1457.7941" adv=".278"/>
        <g unicode="/" glyph="slash" x="317.01094" y="1457.7941" adv=".278"/>
        <g unicode="/" glyph="slash" x="320.90293" y="1457.7941" adv=".278"/>
        <g unicode="x" glyph="x" x="324.79493" y="1457.7941" adv=".5"/>
        <g unicode="0" glyph="zero" x="331.79493" y="1457.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="339.57893" y="1457.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="343.47093" y="1457.7941" adv=".278"/>
        <g unicode="y" glyph="y" x="347.3629" y="1457.7941" adv=".5"/>
        <g unicode="0" glyph="zero" x="354.3629" y="1457.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="362.1469" y="1457.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="366.0389" y="1457.7941" adv=".278"/>
        <g unicode="x" glyph="x" x="369.9309" y="1457.7941" adv=".5"/>
        <g unicode="1" glyph="one" x="376.9309" y="1457.7941" adv=".556"/>
        <g unicode="," glyph="comma" x="384.7149" y="1457.7941" adv=".278"/>
        <g unicode=" " glyph="space" x="388.6069" y="1457.7941" adv=".278"/>
        <g unicode="y" glyph="y" x="392.4989" y="1457.7941" adv=".5"/>
        <g unicode="1" glyph="one" x="399.4989" y="1457.7941" adv=".556"/>
    </span>
</fill_text>

A couple of questions:

1. What rules lead to labeling a set of lines as a block? The documentation points out that 'Multiple text lines are joined via line breaks.'.
2. Is the labeling of a block done by PyMuPDF separately, or, this is a part of the MuPDF libraries' output?

[JorjMcKie]

Establishing the text hierarchy for the TextPage object (which is the basis for all of the .get_text() variants) is done by MuPDF itself.
They use a complex set of heuristics to combine characters to spans, spans to lines and lines to blocks, taking into account fontsize(s) font characteristics like individual character widths / heights, width of the (white) space character(s), replacing overly large distances by spaces, etc, etc.

[qwertynik]

Ok, so it is MuPDF handling the grouping.

Thanks for the clarity.

[JorjMcKie]

Compare file1 and file2 via text extraction, I hope this sheds some light on the above.
The second file has been made by rewriting the characters of the first file in random order.

[qwertynik]

@JorjMcKie
Yes, the differences now are more clear. In the mutool trace output, noticed the letters to be jumbled. So, there's another layer of processing that MuPDF does which leads to a more cleaner output when using the page.get_text('dict') methods.

[qwertynik]

Attaching the images for reference. bboxes from get_texttrace() are highlighted.

First pdf output:

Second pdf output: