Error Creating Vectorstore with PDFs Containing Graphics

[TheCarBun]

The app encounters an error when processing PDFs that contain graphics (e.g., images, charts, or diagrams) during the creation of the vectorstore. This prevents users from uploading and querying such PDFs effectively.

Steps to Reproduce:

1. Open the app and upload a PDF that contains graphics (e.g., scanned documents, PDFs with embedded charts or images).
2. Wait for the vectorstore creation process to begin.
3. Observe the error message or behavior (failure to create vectorstore).

Expected Behavior:

The app should handle PDFs with graphics gracefully by:

• Ignoring non-text content while extracting and processing text.
• Successfully creating a vectorstore from the textual portions of the document.

Actual Behavior:

The app fails to create a vectorstore, likely due to an inability to process graphic elements in the PDF.

Possible Cause:

This issue might be due to the document loader (PyPDFLoader or similar) attempting to process non-text elements like graphics, which are not supported or cause parsing errors.

Proposed Solution:

• Add functionality to detect and skip non-text content during PDF processing.
• Implement robust error handling to ensure the app continues processing valid parts of the PDF while ignoring problematic sections.

[pakagronglb]

I think I found the solution. Let me know if it works on your end. Will send you a PR now :)

[TheCarBun]

The current used solution is to avoid PDFs that contain images.

It would be great if we can use OCRmyPDF to read the PDFs that contain images rather than skipping them.

I'm keeping this issue open.