Why the file size became huge after I write ocr text to pdf?

[nissansz]

dianfei.pdf

dianfei_overlay.pdf

I tried to read pdf pages with dpi=300, then to get text, and put text to original pdf, but the saved file size is too large, 1MB->13MB
how to improve?

tw = fitz.TextWriter(page.rect)
tw.append((int(left), int(bottom)), text, fontsize=(int(bottom) - int(top))*0.7, font=font, small_caps=True)
# tw.write_text(page)
tw.write_text(page, opacity=None, color=color, morph=None, overlay=True, oc=0, render_mode=0)

# doc.ez_save(new_file)
doc.ez_save(new_file, garbage=3,deflate=True)

[JorjMcKie]

Method Document.ez_save already has the defaults garbage=3 and deflate=True.
With TextWriter class, all fonts are file-based and hence embedded into the file.
As Asian fonts are large (and automatically pulled in when necessary), the file size increases - sometimes dramatically as in your case.
Please use doc.subset_fonts() right before saving. This method builds subsets for all eligible fonts: each font is replaced with a version that only contains the glyphs that are actually in use in the PDF.

[nissansz]

Thank you. It works.
Another question is how to better display ocr result on pdf.
Sometimes, the original textline is too dense, it is hard to read ocr text on pdf.
Any suggestions?

[JorjMcKie]

original textline is too dense

Does this mean that the re-inserted, OCRed text is shorter than its corresponding image?

[nissansz]

You can see above image, I need an effect to easily compare result and original text

When too many texts are mixed in an area, it is not easy to read and compare. Any other option for better visualization for comparison?

[JorjMcKie]

You can define a matrix that "streches" the inserted text horizontally. For example if the original text has width w0, then make sure that the new text also has that width. This can be achieve by a scale matrix fitz.Matrix(beta, 1). If we call the insertion point "pos", you can use morph=(pos, matrix).
Compute beta as old_length / new_length.

[nissansz]

Any example effect?

[JorjMcKie]

doc=fitz.open()
page=doc.new_page()
page.insert_text((100,100), "This is some text.")
pos=fitz.Point(100,120)
mat=fitz.Matrix(2,1)  # double the width
page.insert_text(pos, "This is some text.",morph=(pos,mat))
doc.save("x.pdf")

Gives you this:

[JorjMcKie]

accordingly for TextWriter

[nissansz]

If change font to larger, it will be more messy.
Any other color options to solve the demand?

This effect is good.

This effect is not good.

[JorjMcKie]

That is why I mentioned the morph option: it allows you to control the horizontal width only - without increasing the font size.
You can also at the same time shrink the height - again without interfering with the font size: fitz.Matrix(2, 0.5) doubles the width and halves the height.

As per color: each TextWriter can only output one color. To juggle with multiple colors requires a separate TextWriter per color.

[nissansz]

How to convert RGB color to pymupdf color code?
for example RGB=(0,255,0)

[JorjMcKie]

Multiple ways:

1. fitz.sRGB_to_pdf(0x00FF00) = (0, 1, 0)
2. (0, 255, 0) => (0 / 255, 255 / 255, 0 / 255)

[nissansz]

How to add a light blue background 50% transparent to the text I wrote on pdf?

[JorjMcKie]

or fill=fitz.pdfcolor["lightblue"]

[nissansz]

After I created a ocr version pdf, how to remove all ocr result and convert it back to original pdf?

[JorjMcKie]

Use redaction annotations.

1. define a page-wide redaction: page.add_redact_annot(page.rect)
2. apply it, but don't touch the images: page.apply_redactions(images=fitz.PDF_REDACT_IMAGE_NONE)

[nissansz]

What is value and format of rect?

How to construct this rect?
(x, y, w, h)?

[JorjMcKie]

defined via coordinates of top-left and bottom-right points (x0, y0, x1, y1)

[nissansz]

No fill option for textwriter?

tw.append((int(left), int(top)), text, fontsize=(int(bottom) - int(top))*0.7, font=font, small_caps=True)
# tw.write_text(page)
tw.write_text(page, opacity=None, color=color, morph=None, overlay=True, oc=0, render_mode=0)

[JorjMcKie]

no

[nissansz]

Adobe Acrobat will reduce image quality after saving its result.
Any option to maintain image quality?

[JorjMcKie]

Don't know about Adobe here.
Won't happen if you use PyMuPDF's OCR interface.

[JorjMcKie]

or use OCRmyPDF package instead of Adobe

[nissansz]

How to use PyMuPDF's OCR interface? How abotu ocr accuracy?

[JorjMcKie]

1. Install Tesseract. Either permanently set the environment variable described in the installation section, or provide the tessdata folder name in the following.
2. Define an OCR textpage with desired resolution (DPI): https://pymupdf.readthedocs.io/en/latest/page.html#Page.get_textpage_ocr
3. Extract text with the "dict" option: page.get_text("dict", textpage=tp) using the previously created textpage.

[nissansz]

page.draw_rect(rect, color=None, fill=fitz.pdfcolor["lightblue"], opacity=0.5, overlay=False)

TypeError: draw_rect() got an unexpected keyword argument 'opacity'

[JorjMcKie]

https://pymupdf.readthedocs.io/en/latest/page.html#Page.draw_rect

[nissansz]

Used morph and multiple colors. effect seems a little better.
Rect ground color fails
dianfei_overlay.pdf

[JorjMcKie]

Background color is of course an issue. You can only choose between foreground and background - not anything in between, no way.
If you use overlay=False the rectangle inevitably lands behind the image of the page itself.

[nissansz]

draw_rect(rect, color=None, fill=None, width=1, dashes=None, lineCap=0, lineJoin=0, overlay=True, morph=None, stroke_opacity=1, fill_opacity=1, radius=None, oc=0)

correct parameter is fill_opacity=0.5, not opacity=0.5
Now it looks a little better.