repeated saving decreases file size

[cbm755]

I can't share this input file but I can probably prepare a similar one: its just output from LaTeX.

Suppose I copy the pages of src file one at a time to dest file:

src = fitz.open("version1.pdf")
dest = fitz.open()

for n in range(len(src)):
    dest.insert_pdf(src, from_page=n, to_page=n)

dest.ez_save("foo1.pdf")

Background

By default this makes quite large output:

# ls -s 
248 version1.pdf
2304 foo1.pdf  

(my file has 24 pages, likely scales with number of pages).

But this is easy to fix with garbage=4:

dest.ez_save("foo2.pdf", clean=True, garbage=4)

Today's issue

src = fitz.open("version1.pdf")
dest = fitz.open()

for n in range(len(src)):
    dest.insert_pdf(src, from_page=n, to_page=n)

dest.ez_save("foo2-1.pdf", clean=True, garbage=4)
dest.ez_save("foo2-2.pdf", clean=True, garbage=4)
dest.ez_save("foo2-3.pdf", clean=True, garbage=4)
dest.ez_save("foo2-4.pdf", clean=True, garbage=4)

Here I saved the file four times, expected the same output each time. Instead, I get decreasing file sizes:

332 foo2-1.pdf
284 foo2-2.pdf
260 foo2-3.pdf
260 foo2-4.pdf
248 version1.pdf

(I think sometimes I've observed larger files too, not sure).

Question 1

Why does this happen? On one hand, its useful. But OTOH, saving each file "a couple of times" to get a maybe 50% smaller file seems like a poor style choice...

Question 2

Saving seems stateful. Is this desirable? Its unexpected to me: my naive intuition is that saving the file would not change the state of dest in memory: clearly this is not so. Neither save nor ez_save seem to document this behaviour, at least not in the top-level docstring (dest.save?) where I looked.

So minimum fix is probably to document this behaviour. "Note saving a full can change the in-memory data as well."

[cbm755]

I spend a few minutes trying to build a shareable MWE. So far, the effect is less dramatic:

160K version1.pdf
500K foo1.pdf
180K foo3-1.pdf 
172K foo3-2.pdf 
168K foo3-3.pdf 
168K foo3-4.pdf 

(but still noticeable). It seems necessary to have tikz figures and maybe embedded png/eps etc. So perhaps the document needs to be quite complex. The original example was also created on "Overleaf" and I don't know how similar that setup is to my local Fedora 37 with TeXLive.

---

So if needed, I will later create a MWE that is shareable: for now I'm nervous about messing up and sharing the wrong thing.

[JorjMcKie]

If you use clean=True then this will always change appearance streams, because MuPDF cannot / does not check whether or not an appearance stream (like a /Contents object) has already been "cleaned" before.
So in order to do an analysis we would need an experiment with clean=False, deflate=True. Only that indeed would be worthwhile investigation.

[JorjMcKie]

Overall I think we have no issue here. I have made several experiments without using the clean=True argument, which all lead to a constant filesize, or even the identical file (apart from changing /ID in the trailer).
So I hope you are okay to re-labelling as a "Discussions" item.