Epic: Document Processing

[ps24601]

Tools explored:

1. PymuPDF
2. DIT (https://huggingface.co/spaces/nielsr/dit-document-layout-analysis)
3. Nougat: https://huggingface.co/spaces/ysharma/nougat
4. AXAParser

1,2and 4 have documentation in repo, but Nougat is kept only for backup as replacement for pyteseeract.
We will be using the Llamaindex instead of Haystack. Some reference code can be found here: https://github.com/MoritzLaurer/rag-demo.
Llamaindex has inbuilt few parsing methods: https://huggingface.co/spaces/m-ric/chunk_visualizer

Task 1: Experimentation with Llama Index, need to explore what components are there in llamaindex for doc processing and good they perform.
Task 2: Read PDF file, Detect image vs PDF, IF normal PDF remove images from pdf file. Use AXA Parser to get different entities Normal text, Tables, Heading and save raw corpus with meta data. Write Table as json with header values as metadata (Explore if Llamaindex utils an be used in this regard)
Task 3: Use OCR tool to extract text especially Nougat or DIT.
Task 4: Use Llama Index to create the database

[jonas-nothnagel]

The appropriate documentation for LamaIndex can be found here: https://docs.llamaindex.ai/en/stable/module_guides/loading/ingestion_pipeline/root.html

Still question remain on what embeddings and what vector storage to use. These decision shall inform the choice of framework and not the other way around.

Overview of vector databases: https://docs.llamaindex.ai/en/stable/module_guides/storing/vector_stores.html