Detecting issue with Font in a PDF

[qwertynik]

@JorjMcKie

The below snippet of code was shared by you earlier. While this manages to extract the font for most cases, there are cases when the text rendered in a PDF using the extracted fonts is jumbled.

fitz.TOOLS.set_subset_fontnames(True)  # return subset fontnames
fontitems = page.get_fonts()  # list of all fonts
fonts = {}  # fontname -> fitz.Font
for f in fontitems:
    if f[1] == "n/a":  # not embedded: Helvetiva -> fitz.Font("Helvetica")
        if f[2] == "Type3":
            raise RuntimeError("cannot handle Type 3 fonts")
        fonts[f[3]] = fitz.Font(f[3])
    else:  # extract the buffer and make a font of it
        fname = f[3]
        buffer = doc.extract_font(f[0])[-1]
        font = fitz.Font(fontbuffer=buffer)
        fonts[fname] = font

Any ideas to detect errors in the font extraction?

[JorjMcKie]

I would need an example.
There are cases, when the fontname returned by get_text("dict") is hidden somewhere in the PDF font definition, i.e. several connected xrefs jointly describe one font, but may contain (slightly) different names for that same font.
If you look at the details of the font replacing scripts, you will find code that tries to trace this situation - should be in repl-fontnames.py.

[qwertynik]

Had an example, but do not have access to it now. Will check the repl-fontnames.py script for more ideas on this and revisit this discussion.

[qwertynik]

@JorjMcKie
Went through the script repl-fontnames.py, did not find anything in particular that attempts to detect font errors in a PDF.

[JorjMcKie]

If this happens, then already the text extraction from the source page delivered invalid characters 0xFFFD. This may happen when the font is unsupported by MuPDF, or contains inbuilt protection against extraction, etc.
Except for "punctual" OCRing text spans containing those character codes (there is a script) I see no way out.
The font replacer scripts indeed contain no precaution for this ...

[JorjMcKie]

where the PDF itself contains the �(0xFFFD) character

That is correct - and may be intentional. You can output text containing this character yourself with PyMuPDF. Detecting real errors in a font is not (Py-) MuPDF's business however, and in most cases not distinguishable from the other mentioned situations. There is font software specialized on these things.
This also is of no interest for your use case if I understood well.

[JorjMcKie]

Also, the font may be or have been okay before insertion, but some of its component may have been deleted / damaged in the PDF due to some other error.
Create this type of situation like so:

import fitz
doc=fitz.open()
page=doc.new_page()
tw=fitz.TextWriter(page.rect)
tw.append((100,100),"just some text")
tw.write_text(page)
print(page.get_fonts())
[(5, 'cid', 'Type0', 'Helvetica', 'F0', 'Identity-H')]
doc.xref_set_key(5, "ToUnicode", "null")  # remove glyph-to-unicode mapping
doc.save("x.pdf")
doc.close()
doc=fitz.open("x.pdf")
page=doc[0]
print(page.get_text())
��������������

But the PDF is still viewable nicely!

[qwertynik]

A great example @JorjMcKie. Makes it all the more tricky. But also shows the robustness or ambiguity (depending upon how one interacts with the PDF 😂) of the PDF specification

[JorjMcKie]

Now you know what to do if you want to immunize your text against extraction - without distroying the appearance (just kidding, there arre better ways!).

[qwertynik]

Haha, yeah! Would have used this approach to make extraction difficult. But since you mentioned there are better ways, will look for them when required.