What does size mean in a span object

[zhichu]

Normally I thought size is the height of the font, but I found it's not the case. I use LaTeX to produce pdf files and if a line is too wide/narrow, it will be shrinked/expanded a little bit horizontally to fit in, making the text matrix something like 0.98 0 0 1 xxx xxx/1.02 0 0 1 xxx xxx. So I got different sizes. It's not even proportional to the x scale. I'm so confused. What I want is just the font size, multiplied by the y scale, i.e., just the height. Can I do that?

[JorjMcKie]

A span dictionary's "size" key indeed is the fontsize. This has nothing to do with the inter-character distance. Whenever text in a line changes its fontsize, a new span is started.
This also happens, if MuPDF's heuristics determine that the gap between text pieces is too wide. This situation may even cause a new line to be commenced. This type of thing can typically be seen in tables.

The vertical distance between lines lies completely in the hand of the document creator. Although the font properties ascender and descender are usually taken to compute the line height via (ascender - descender) * fontsize, this is not mandatory.

Taking the example font Helvetica, you will see the values:

In [1]: import fitz
In [2]: font = fitz.Font("helv")
In [3]: font.ascender, font.descender
Out[3]: (1.0750000476837158, -0.29899999499320984)
In [4]: fontsize = 10
In [6]: (font.ascender - font.descender)*fontsize
Out[6]: 13.740000426769257
In [7]:

So the standard line, span and each character's height is 13.74 for fontsize 10 in this font.
If multiple fontsizes are used in the same line, the line bbox height will obviously be the maximum of its span heights.

The MuPDF standard to compute a characters height is also (ascender - descender) * fontsize. This is for example taken to determine if a character intersects a redaction rectangle, but also the hit rectangles of page.search_for() use it.

While PyMuPDF cannot influence the logic internally used in redaction logic, you can force almost all other text-related logic to use the fontsize as character / span / line height: fitz.TOOLS.set_small_glyph_heights(True).
This global parameter will cause text searching and text extraction to report fontsize as the bbox height.
It has however no influence on (X)HTML / XML output.

[zhichu]

Wow, that was quick.

I'm not sure I made myself clear. I attached a pdf file created by LaTeX to explain again.

If you open the pdf file with a text editor, you can see the two lines (around line 10) are:

BT
/F46 9.96264 Tf
1 0 0 1 66.656 729.828 Tm [<00690040006E0063002E>-250<0042>-250<002C001D004C0033>-250<006A0052>-250<006A00400033>-250<002C0052004E002C0049006E006300430052004E>-250<006A0040001D006A>-250<006A00400033>-250<0030003300630043003C004E00330061>-250<00520038>-250<001D>-250<004E0033>]TJ
0.986 0 0 1 314.687 729.828 Tm [<00690040006E0063002E>-253<0042>-252<002C001D004C0033>-253<006A0052>-253<006A00400033>-253<002C0052004E002C0049006E006300430052004E>-253<006A0040001D006A>-252<006A00400033>-253<0030003300630043003C004E00330061>-253<00520038>-253<001D>-253<004E0033>30<0073>]TJ
0 g 0 G
0 g 0 G
ET

The first line misses a w so the text matrix is an identity matrix 1 0 0 1 66.656 729.828 Tm, the second one is a little too wide so it's shrinked 0.986 0 0 1 314.687 729.828 Tm. But the font is still the same: /F46 9.96264 Tf. But the s['size'] says otherwise:

 $ ipython
Python 3.10.5 (main, Jun 10 2022, 03:11:15) [GCC 11.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import fitz

In [2]: doc = fitz.open("test.pdf")

In [3]: page=doc[0]

In [4]: blks = page.get_text("dict")["blocks"]

In [5]: b=blks[0]

In [6]: b['lines'][0]['spans'][0]['size']
Out[6]: 9.962639808654785

In [7]: b['lines'][1]['spans'][0]['size']
Out[7]: 9.892655372619629

Is there anything I can use to get 9.96264*1? You can see even the first line does not give the "accurate" result, but close enough. For the second line: 9.96264*0.986=9.82316304 but 0.986**.5 * 9.96264=9.892655711628986, almost there. That means the size is fontsize scaled by the square root of x scale times y scale. I don't think this value would be very useful. My proposal is, may I also have the text matrix information, so that I can manipulate the values as I wish.

test.pdf

[JorjMcKie]

Ah, got you.

The value reported in span["size"] obviously must be the actually used fontsize in the span. No one ever said it is guaranteed to be the value in command Tf.
There is no way (in PyMuPDF) to access the most recently used Tm separately, but you could use mutool trace output for that.
It is some XML format, watch out for the trm parameter within here:
test.zip

[JorjMcKie]

BTW, the bbox heights of both spans are the same:

for b in page.get_text("dict",sort=True)["blocks"]:
    for l in b["lines"]:
        for s in l["spans"]:
            bbox=fitz.Rect(l["bbox"])
            print(bbox.height, s["size"], s["text"])

            
13.31008529663086 9.962639808654785 Thus, I came to the conclusion that the designer of a ne
13.31008529663086 9.892655372619629 Thus, I came to the conclusion that the designer of a new

If you use fitz.TOOLS.set_small_glyph_heights(True) you get this:

fitz.TOOLS.set_small_glyph_heights(True)
True
for b in page.get_text("dict",sort=True)["blocks"]:
    for l in b["lines"]:
        for s in l["spans"]:
            bbox=fitz.Rect(l["bbox"])
            print(bbox.height, s["size"], s["text"])

            
9.962638854980469 9.962639808654785 Thus, I came to the conclusion that the designer of a ne
9.892658233642578 9.892655372619629 Thus, I came to the conclusion that the designer of a new

[zhichu]

Thanks! That's what I wanted... for now.

[JorjMcKie]

😊 good, but you will never ever get more anyway from PyMuPDF 😉.

[JorjMcKie]

Your example is definitely an interesting one!
So I should thank you for this!
I made a few cross calculations. The font used we have a standard line height of ascdsc=s0["ascender"]-s0["descender"] = 1.3359999358654022.
If we divide the bbox height by this value,

In [16]: bbox0.height / ascdsc
Out[16]: 9.962639173338859

Which is the fontsize of the Tfcommand.
The first span almost exactly has this as its "size":

In [20]: bbox0.height / ascdsc - s0["size"]
Out[20]: -6.353159260896746e-07

But the second span gives us this - a quite notable deviation:

In [21]: bbox1.height / ascdsc - s1["size"]
Out[21]: 0.06998380071923016

From this you can calculate all you need I suppose.

[zhichu]

I think I got how these numbers were calculated here. The text font size is 10pt, but this is in LaTeX: 72.27pt = 1in. So in PDF unit system (where 72pt = 1in), the font size is

10 * 72 / 72.27 = 9.962640099626402

Let's say the text matrix is

a b 0
c d 0
e f 1

s["size"] is calculated by multiplying the square root of the determinant of the x,y part: a d - b c, or probably just a d, I will do some experiments later.

[buptyyf]

The unit of span size is pt not px?

[JorjMcKie]

The unit of span size is pt not px?

This is points in PDF user units.

MuPDF computes the font size as the geometric mean: sqrt(trm.a * trm.d) of the current text matrix.