Allow embedding typst source into resulting pdf

[iilyak]

the draw.io application is able to store the source of the diagram inside the so called editable png. This means the draw.io application can edit the resulting diagram. Something similar could be done with pdf. Since pdf support custom metadata. The idea is:

1. to store the source used to produce the pdf in the metadata
2. if queries are used in the source inject the results produced by the queries in the metadata
3. teach typst to extract source from pdf
4. teach typst to run queries (the same as used for construction) against resulting pdf

This would be beneficial for things like journals.

Let's say I would implement the function to simplify writing journal entries. These entries would be stored in the context (here is an early experiment which demonstrates how I plan to store the entries). Then I would use queries to extract any entry given its id or date. I would like to be able to write rust application which would be able to query resulting pdf (using typst as a library).

[eduardz1]

The good news is that embedding files is now supported on main after #5221

Implementing your proposal would just be a matter of extending the Typst compiler to accept also files with the .typ.pdf extension perhaps, like drawio does with their .drawio.svg extension and just look at the embedded Typst code, tough incremental compilation might not be as straight forward

[Kreijstal]

why extend compiler? just write a tool extracttypstfrompdf file.typ.pdf out.typ, dont overengineer it

[iilyak]

why extend compiler? just write a tool extracttypstfrompdf file.typ.pdf out.typ, dont overengineer it

For the context:
In my use case I am planing to use it in school setting (9-13 y.o.). The students would be asked to document their research project activity in the journal. Since the template would be consisting of multiple files I was thinking the following workflow would be easier for kids than messing with additional tools or archives.

1. install typst
2. download typst template distributed as <name>.typ.pdf
3. run typst command to extract source from <name>.typ.pdf file (there would be a helper to run this)
4. work on the journal
5. run command to render it as <name>.<version>.typ.pdf (there would be a helper to run this)
6. send rendered pdf to teacher.

However to your point. Since I would need to distribute the helpers I could instead (as you suggest) distribute a custom app which embeds typst.

[weirdrock]

I've gotten this working using the pdf.embed function like so

// File name: source.typ
= A header
Lorem ipsum

#pdf.embed(
  "source.typ",
  relationship: "source",
  mime-type: "text/plain",
  description: "Document Source Code"
)

This works great for single file documents, but it breaks down if you start to use imports.

Notably, the embed function supports a bytes parameter to use instead for the content of the embedded file. Has anyone found a way to get a bytes object of the concatenated source, which can be passed to embed?