insert_textbox prunes whitespace

[Bonnevie]

I need to copy spans of text from an existing pdf to a new one, and hoped to rely on insert_textbox, by first identifying the appropriate bounding box by using get_text("dict") on the original page and using it as an argument for the insertion command.

My problem is that my text comes with encoded whitespace and the insert_textbox command seems to prune whitespace characters and also tightens the bounding box. Oddly, this is not the case for insert_text, but there does not seem to be a straightforward way to get the point of insertion needed for that command from the available get_text("dict") span metadata.

In short, is there a way to insert trailing whitespace with insert_textbox?

I'd like for this to be a one-to-one copy so that the documents are encoded as similarly as possible. This also causes problems if I do a re-extraction of the new document, since whitespace will have to be inferred.

[JorjMcKie]

Method insert_textbox() is bad idea, because it contains so much logic to prepare for fitting the text in the current rect.
Use insert_text. The insertion point is like always span["origin"]. Obviously you must ensure to use the same font, fontsize, text color, writing direction / transformation as the original text.
Doing all this automatically should take care of the right text lookalike.

I am going to convert to a "Discussions" item.

[Bonnevie]

Thanks for the fast response @JorjMcKie ! and a great tool :)

ah, span["origin"] will help me a lot, didn't realize that it was the point of insertion.

Trying to use insert_text, I run into a new issue that my normal whitespace seemingly gets converted to non-breaking spaces? This only happens for some fonts, like Calibri, but the text was extracted with that font as well. I have extracted the font and then inserted it in the new document - encoding switches from WinAnsiEncoding to Identity-H, but otherwise it appears to work as intended - except for that whitespace issue.

edit: just tried inserting on the page I'm extracting the font from, and there it works fine, so problem seems to arise from the extraction->insertion operation. I'm on linux and the document is likely generated on Windows if that matters.

[JorjMcKie]

Re your edit:
That indeed looks like font re-insertion is causing this effect, and PyMuPDF is not.

[Bonnevie]

I'm using the default flags, but I do get the whitespace in the extraction, it just gets converted to non-breaking space when I run insert_text. I also have the same issue if I simply define a python string with a space and try to insert that, so I don't think the extraction process can impact it.
The font is a subset of Calibri Bold, and I tried saving it to file and then loading with the fontTools package, so I can verify that the CMAP contains both space characters (breaking space 32 and non-breaking 160). They both map to the glyph with name space.

Can the encoding change of the font be avoided so that the reinserted font is 1:1 with the original?

[JorjMcKie]

ok, understand

Can the encoding change of the font be avoided so that the reinserted font is 1:1 with the original?

No, this happens under the hood and is done by MuPDF.

[Bonnevie]

I tried instantiating a Font object with the TTF, and unicode_to_glyph_name maps 32 to glyph spacehackarabic and \xa0 non-breaking space to nonbreakingspace. I can also affirm that TextWriter produces the same error. glyph_name_to_unicode also maps spacehackarabic back to ordinary space (32).
Doesn't this mean that there actually is a bug somewhere?

[JorjMcKie]

Could be ... if so, then either in MuPDF, or in the font itself.
You could submit a bug to MuPDF's issue system here.