Page break functionality, respecting links across multiple pages

[Julynx]

I'm making a script to translate Markdown into HTML, and then pass the HTML + some CSS to a fitz Story, place that Story to a document, and save the document as a PDF.

This is the relevant part of the code:

        rectangle = fitz.paper_rect("A4")
        position = rectangle + (36, 36, -36, -36)
        html_content = markdown_parser(page,
                                       extensions=list(self.plugins))
        story = fitz.Story(html=html_content,
                           user_css=self.custom_css,
                           archive=".")

        while True:
            device = output_writer.begin_page(rectangle)
            pages_left, _ = story.place(position)
            story.draw(device)
            output_writer.end_page()

            if pages_left <= 0:
                break

This works perfectly fine, except that hyperlinks (both to the same document and to URLs on the internet) are not clickable in the final PDF.

I have tried Story.write_with_links() but I'm getting errors in the terminal that the destination of the link does not exist, plus I cannot manage to get the margins to be the same as with the looped approach in the code above.

I'm using the 'markdown' package to get the HTML from the MD file that I then pass to fitz, if that helps.

PyMuPDF==1.23.3

[Julynx]

I already got it to work. However, now I need to add support for page breaks.

I managed to achieve that with this code, but sadly this breaks links between different pages:

markdown_parser = markdown.markdown
pdf_document = fitz.open()

content_parts = re.split(r'<pagebreak>', self.content)

for part in content_parts:
    html_content = markdown_parser(part, extensions=list(self.plugins))
    story = fitz.Story(html=html_content, user_css=self.custom_css, archive=".")

    def rectfn(rect_num, filled):
        rectangle = fitz.paper_rect("A4") + (36, 36, -36, -36)
        return fitz.paper_rect("A4"), rectangle, None
    
    document = story.write_with_links(rectfn)
    pdf_document.insert_pdf(document)

pdf_document.save(file_path)
pdf_document.close()

Notice that, because story.write_with_links(rectfn) is called for each story or page sepparately, if it contains links to another story or page, those links will not be found and the program will fail:

RuntimeError: No destination with id=quotes, required by position_from:  depth=1 heading=0 id=None rect=(420.82794189453125, 196.47598266601562, 457.5119323730469, 208.47598266601562) text=None open_close=3 rect_num=1 href='#quotes' page_num=1

Is there a way to mantain this page break functionality and links between pages working at the same time?

[JorjMcKie]

Cool stuff, congratulations so far!
I have no solution for the remaining problem out of the hip. But I assume you need to use the position recorder - potentially the ones of each part-story to communicate with each other (via global variables).
I need to lookup how to specify element position recorders with the write_with_links() method.

[Julynx]

Thanks! I have tried simply doing this:

self.content = self.content.replace("<pagebreak>",
                                    '<p style="page-break-after: always;"></p>')

markdown_parser = markdown.markdown
html_content = markdown_parser(self.content,
                               extensions=list(self.plugins))

story = fitz.Story(html=html_content,
                   user_css=self.custom_css,
                   archive=".")

def rectfn(rect_num, filled):
    rectangle = fitz.paper_rect("A4") + (36, 36, -36, -36)
    return fitz.paper_rect("A4"), rectangle, None

document = story.write_with_links(rectfn)
document.save(file_path)
document.close()

However, in that case only the first page will be written to the PDF. For some reason, it looks like story.write_with_links(rectfn) will stop writing if it finds '<p style="page-break-after: always;"></p>' in the HTML, because when I remove that, it renders all pages.

I still need to find a way to render all pages with links and page breaks if that's possible.

[JorjMcKie]

Interesting.
If you use a element position plug-in, you could insert nodes with a special, recognizable id pattern (instead of that "page-break-after: always;", upon when encountered, the plugin sets a global variable like "PAGEBREAK=True`. Your rectfn should test this value and react appropriately.

[JorjMcKie]

id's must be unique of course, so you must think about a pattern like id="pagebreak-%i" or something.

[Julynx]

I don't really have a clear path on how to make an implementation that follows those ideas.
I'm still getting used to the library and in the process of learning a lot of things.

Still, thank you for the valuable insight, I will update this if I'm able to reach a solution.

[JorjMcKie]

I don't really have a clear path on how to make an implementation that follows those ideas. I'm still getting used to the library and in the process of learning a lot of things.

Still, thank you for the valuable insight, I will update this if I'm able to reach a solution.

Oh, I am sure you will be successful.
If possible, a contribution to our utilities repo would be welcome ...😎

[Julynx]

I would love to see an easy way to make the Story.write and Story.write_with_links methods jump to the top of the next page of the pdf and keep writing from there when they encounter a '<p style="page-break-after: always;"></p>' in the HTML.

At the moment, it looks like they stop writing altogether when that happens, as for HTML content with multiple page breaks ('<p style="page-break-after: always;"></p>'), only the first "page" is actually saved to the PDF document. I made an issue for this.

I think I have a lot to learn before I'm ready to contribute to a project like this, but I'm looking forward to that moment!