deflate and garbage options not compressing file enough

[aarthihWN]

Hi Jorj,
Need your assistance again.
I am saving a bunch of pdfs after adding page numbers, with these options - clean=True, garbage=3, deflate=True. The file size of a few files is still large. For example, a file that was originally 2.3MB is 3.6 MB after saving. Would like to know if there is a way to bring down the file size? As the no. of files is high, the cumulative increase adds up significantly.
Unfortunately, I cannot share the file but is there any other way I can get you the data about the file?
You help is appreciated. Thanks. Below is the code:
doc = fitz.open(path)
for number,page in enumerate(doc):
page.clean_contents()
page.insert_text((page.rect.width - 60, page.rect.height-20), str(page_number_for_print), fontsize=18)
page.clean_contents()
doc.save('something.pdf',clean=True,deflate=True, garbage=3)
doc.close()

[JorjMcKie]

The save options are ok - you might want to use garbage=4, which also removes any binary stream duplicates.
Other than that ...😒
MuPDF unfortunately does not yet support all PDF compression options (specifically /ObjStm object streams).

A possible way out is not saving to a new file, but incrementally. Then only the update deltas are appended to the file - existing stuff remains untouched. This has a few consequences though:

1. page.clean_contents() always makes a new page /Contents stream. This will bloat the data delta when saving incrementally - a simple NO GO.
2. There is page.wrap_contents() (written by me 😉) which adds 2 small /Contents objects: q before and Q after the existing contents - leaving everything else intact and maintain a small incremental delta. It should work in most cases as well: If existing contents in total have equal numbers of q and Q instructions, you should be safe. This mostly is the case.

So using incremental saves and using wrap_contents() instead of clean_contents() will cause change deltas to be appended. The deltas will contain the page number text, the q/Q stuff ... and the font used for the page number text. So it will be vital to use a small font! This is true if you use page.insert_text and one of the Base-14 fonts (Helvetica, Times Roman, Courier). They have no font files - and are represented by a few ASCII text descriptions only.

[aarthihWN]

Hi, thanks for your reply.
I was using saveIncr() earlier but switched to save() assuming that to be the cause of the problem. Little did I know it was the combination of both.

However, I have had to use clean_contents() because of another issue. See here - #2152
Using clear_contents() solved a lot of problems. I am not very good with the pdf internals. Would changing it to wrap_contents() bring back the earlier problem?

[JorjMcKie]

Would changing it to wrap_contents() bring back the earlier problem?

(Very) probably not. That is why I was bringing this up. I would guess that in more than 9 of 10 cases it will address the wrong position issue.

[aarthihWN]

wrap_contents() works better than the previous option for most files. A few, however, seem to do well with the deflate-garbage option. Overall, wrap_contents() seems to do the job.

Jorj, thanks again for helping out. Thanks for all that you do with pymupdf.