Why is the logic for calculating the number of ‘#’ like this?

[Fianax]

I am creating my own hdr_info_custom for the calculation of the sections.

But I wanted to know how do you do this calculation, i.e. how do you calculate the number of ‘#’?

I've been looking at the code and I got this:

fontsizes = {}
for pno in pages:
    page = mydoc.load_page(pno)
    blocks = page.get_text("dict", flags=fitz.TEXTFLAGS_TEXT)["blocks"]
    for span in [  # look at all non-empty horizontal spans
        s
        for b in blocks
        for l in b["lines"]
        for s in l["spans"]
        if not is_white(s["text"])
    ]:
        fontsz = round(span["size"])
        count = fontsizes.get(fontsz, 0) + len(span["text"].strip())
        fontsizes[fontsz] = count

Why, in order to count the number of times a word of size X appears, do you count the size of the string, i.e. the number of characters?

Why don't you just count the number of words with that X size?

[JorjMcKie]

Word from get_text("words") are made from consecutive characters ... of whatever font, font size, etc.
Text extracted by to_markdown() extracts spans - which never mix up text particles like this.
Therefore I needed the font size and not the rectangle height of a word.
If you want to fall back to rectangle heights, this roughly probably also works. Use word heights and hope that the span["bbox"] is sufficiently close ...

[JorjMcKie]

And don't forget:
The whole approach has basic issues. E.g. it doesn't look at differences between text styles and or weights. Many headers have the same font size as the following text - but are bold or italic or different color etc.
All that is ignored by my simplifying approach.

Maybe, if the file has a TOC, the "title" items and their hierarchy levels deliver much better approximations. Then the first span of a line would have to be compared to the bookmark items pointing to that page.
Hope I am clear.

[Fianax]

And what happens when there are several words of different sizes in the same sentence?

[JorjMcKie]

different spans

[JorjMcKie]

the first span decides about header level

[Fianax]

So in the same blocks[x][‘lines’] there can't be spans with different sizes?

Or in the same blocks[x][‘lines’][y] all spans have to have the same size?

blocks = page.get_text("dict", flags=pymupdf.TEXTFLAGS_TEXT)["blocks"]

[Fianax]

but if for example you have this sentence: hello world HOW ARE YOU

the lower case part has a font size of 12 and the upper case part has a font size of 16.

You say they are different spans but when creating the sentence, which size is used, 12 or 16?

If size 12 is the predominant size in the document, in this case this sentence will have ‘#’ or not?

In pdf4llm==0.0.7 it would have ‘#’ but in 0.0.9 it would not.

[JorjMcKie]

The whole line will be treated as body text because span[0] has body text font size.

[Fianax]

So this is possible?

These two spans are on the same line, so there are two different font sizes on that line. Which one is chosen when this happens?

prueba_sentence_distinct_size.pdf