Create a new PDF object from scratch

[angea]

Is your feature request related to a problem? Please describe.
When I merged documents, I noticed that the /Info dictionary from the trailer is lost.

I noticed it's possible to copy it from the source file:

ref = old.xref_get_key(-1, "Info")[1]
ref_nb = int(ref.split(" ")[0])
info_dict = old.xref_object(ref_nb)

Then add it manually in the trailer of the new file:

new.xref_set_key(-1, "Info", info_dict)

Describe the solution you'd like
Instead of adding it to the trailer, I'd prefer to create a new object for that - like it was initially - but xref_set_keys` doesn't suit my needs.

[angea]

I realized that set_metadata could do the trick.

Is there a standard way to create new objects ourselves though ?

[JorjMcKie]

I realized that set_metadata could do the trick. Is there a standard way to create new objects ourselves though ?

This is the standard way: whether or not there was a metadata before, set_metadata will create one or overwrite one or whatever. You are also free to choose information from either source file, or mix those.

[JorjMcKie]

"set_metadata since some versions does no longer create empty keys."

Well, this may be misleading:

• set_metadata({}) removes the metadata completely, including the object itself
• e.g. something like set_metadata({"title": "something"}) will set all other standard metadata keys to the PDF equivalent of Python None, which is null: <</Author null/Producer null/Creator null/Title(something)/CreationDate null/ModDate null/Subject null/Keywords null/Trapped null>>. The PDF spec says, that keys not set should not be set to empty (PDF) strings ().

The reason for this is that the metadata is not necessarily a separate xref: I have seen situations like /Root 1 0 R /Info 1 0 R in the PDF trailer. In this case the metadata's single keys are 1st level keys in the catalog, because the /Info object - in contrast to /Root - has no own PDF /Type (/Type /Metadata is reserved for XML metadata).
I am not even sure whether such cases are 100% ok as per PDF spec ... anyway, it does happen and it is not prohibited.
To update the metadata in every possible situation, you must set each single standard metadata key to something - and be it null.

[angea]

Ah, TIL again, thanks. I see null rarely and too many PDFs with () as empty metadata, I guess.

[angea]

BTW what about just not writing at all the keys with a null value in the /Info dictionary ?

So in your example, just <</Title (something)>>.

[JorjMcKie]

BTW what about just not writing at all the keys with a null value in the /Info dictionary ?

How would you differentiate between the intention to (1) remove a key, and (2) leave it unchanged as it is?
Remember, that doc.metadata as a Python dictionary always contains all standard keys - that fact should not be changed, because users are accustomed to using it as a template for their changes.
Your suggestion amounts to actively delete a key from that dictionary if it should not be changed - which would mean a change in current behavior.
And this to gain ... what? A few bytes more or less with zero functional impact?

[JorjMcKie]

What I really want to consider is a change, that if a key is really absent in the agument dict, then I won't generate it with a null entry.

[JorjMcKie]

Your wording is somewhat ambiguous:
If you were asking whether you can create arbitrary other objects beyond metadata, the answer is also yes, for pretty much everything.

[angea]

So what's the way to create new object from scratch? I know about xref_set_key to add a key to an object, but no way to add my own object to a PDF.

[JorjMcKie]

When I merged documents, I noticed that the /Info dictionary from the trailer is lost.

This is not the right way to describe what is happening when joining PDFs.
Presuming you were talking about method Document.insert_pdf(), then what is really happening is that the issuing document has the "lead": it will be enriched with pages from another document ... and only pages, nothing else. The issuing document retains its metadata, its table of contents, any embedded files and what not.
You can of course lookup other information from the second document and make them somehow also part of the target of the merge, but this requires more than what insert_pdf() can do.

[angea]

I see, it makes sense.

TBH I was abusing this feature to make it possible to get the object 1 unused, at the start of the file, as an easy way to embed data in a PDF at a predictable offset.

[JorjMcKie]

There is a plethora of features to discover in pymupdf.
Among them the full range of PDF's EmbeddedFile handling - to be used for all sorts of data (other PDF programs, Excel files, what not) and retrieve them by name.
The pymupdf docu is a 400 "Letter" pages book, so there is a lot ...!

[JorjMcKie]

So what's the way to create new object from scratch? I know about xref_set_key to add a key to an object, but no way to add my own object to a PDF.

Fairly straightforward I hope.

1. xref = doc.get_new_xref() - just defines a new xref number
2. doc.update_object(xref, "<< object definition >>") - fills in definition
3. if it should be a stream object: doc.update_stream(xref, b"stream data", new=True)