Merging multiple DoclingDocument objects

[triaeiou]

Hi,
Are there any standard methods to merge multiple DoclingDocument objects other than rolling my own? I find that on my setup the docling process crashes on large PDF's, however i can split the PDF into smaller chunks and parse each chunk separately. For correct post processing and I would however like to merge the chunks back together to a single DoclingDocument (with correct prov[-1].page_no etc.). Any pointers appreciated!

[cau-git]

@triaeiou We don't have anything yet but I like the idea. Also, I would like to understand on which documents you see docling crash, and on which environment / options? Could you provide us some details or one of the affected PDFs? Thanks!

[triaeiou]

Hi,
I am running docling under windows 11 on an i7-1255U 2.60 GHz with 32GB RAM, no GPU from python with

    converter = DocumentConverter()
    result = converter.convert(Path(src))
    with open(dest, 'wb') as output:
        pickle.dump(result.document, output)

I can't share the PDFs due to copyright restrictions, but something like 3000pg/250MB medical ebook. I say crash but it may be the main or a subprocess that hangs/crashes as the result is that it doesn't finish even if left running for 48hrs. However, when splitting it in 6 parts each part finishes in about an hour,

[dolfim-ibm]

That would indeed be a nice utility. Keep in mind that merging DoclingDocument would have to take care of matching the proper page numbers in a few places (e.g. in the provenance details)