Trouble consuming PDF's downloaded from the internet [Errno 2] No such file or directory

[philosowaffle]

Hello all, I'm really excited about paperless-ng and have recently begun my journey of digitizing my documents. So far I have uploaded ~100 documents on paperless, some have been digital documents but most have been documents I personally scanned. This has gone great so far.

Last night I started trying to upload some PDF device manuals. To save time, I first would search the internet to see if a pdf already existed for my device, if so, download the PDF and copy to the consume directory. For every pdf I've downloaded from the internet, paperless has failed to consume it with the following error:

[Errno 2] No such file or directory: '/usr/src/paperless/src/../consume/E12509_STRIX_Z270H_GAMING_Motherboard.pdf'

Things to note:

1. All other documents have worked fine for me, but I have yet to download one of these manuals from the browser that would work. I will attach two sample pdfs to this thread.
2. While the importer throws the above error. The document is actually imported into the media directory. I see that the document exists in the originals and archive folders, and a new thumbnail is added. But the document never appears in paperless-ng UI.
3. The PDFs seem to be valid format. I can open them and read them, and I ran them through a PDF format checker online.

Deployment:

1. Docker
2. latest tag
3. PAPERLESS_CONSUMER_POLLING=300
4. I checked all the volume permissions, everything looks good
5. Documents I scan myself and add to the consume directory work perfectly

Logs:

[Errno 2] No such file or directory: '/usr/src/paperless/src/../consume/E12509_STRIX_Z270H_GAMING_Motherboard.pdf'

Based on these logs, the whole import went well, but then failed on the Delete step.

[2021-12-02 16:55:13,066] [DEBUG] [paperless.management.consumer] Waiting for file /usr/src/paperless/src/../consume/E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf to remain unmodified

[2021-12-02 16:55:18,075] [INFO] [paperless.management.consumer] Adding /usr/src/paperless/src/../consume/E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf to the task queue.

[2021-12-02 16:55:18,190] [INFO] [paperless.consumer] Consuming E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf

[2021-12-02 16:55:18,209] [DEBUG] [paperless.consumer] Detected mime type: application/pdf

[2021-12-02 16:55:18,228] [DEBUG] [paperless.consumer] Parser: RasterisedDocumentParser

[2021-12-02 16:55:18,237] [DEBUG] [paperless.consumer] Parsing E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf...

[2021-12-02 16:55:29,779] [DEBUG] [paperless.parsing.tesseract] Extracted text from PDF file /usr/src/paperless/src/../consume/E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf

[2021-12-02 16:55:29,911] [DEBUG] [paperless.parsing.tesseract] Calling OCRmyPDF with args: {'input_file': '/usr/src/paperless/src/../consume/E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf', 'output_file': '/tmp/paperless/paperless-hnz68_xx/archive.pdf', 'use_threads': True, 'jobs': 4, 'language': 'eng', 'output_type': 'pdfa', 'progress_bar': False, 'skip_text': True, 'clean': True, 'deskew': True, 'rotate_pages': True, 'rotate_pages_threshold': 12.0, 'sidecar': '/tmp/paperless/paperless-hnz68_xx/sidecar.txt'}

[2021-12-02 16:55:44,863] [DEBUG] [paperless.parsing.tesseract] Incomplete sidecar file: discarding.

[2021-12-02 16:55:56,677] [DEBUG] [paperless.parsing.tesseract] Extracted text from PDF file /tmp/paperless/paperless-hnz68_xx/archive.pdf

[2021-12-02 16:55:56,684] [DEBUG] [paperless.consumer] Generating thumbnail for E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf...

[2021-12-02 16:55:56,693] [DEBUG] [paperless.parsing] Execute: convert -density 300 -scale 500x5000> -alpha remove -strip -auto-orient /tmp/paperless/paperless-hnz68_xx/archive.pdf[0] /tmp/paperless/paperless-hnz68_xx/convert.png

[2021-12-02 16:55:57,254] [DEBUG] [paperless.parsing.tesseract] Execute: optipng -silent -o5 /tmp/paperless/paperless-hnz68_xx/convert.png -out /tmp/paperless/paperless-hnz68_xx/thumb_optipng.png

[2021-12-02 16:56:05,996] [DEBUG] [paperless.consumer] Saving record to database

[2021-12-02 16:56:06,240] [DEBUG] [paperless.matching] DocumentType Statement matched on document 1971-06-01 E12509_STRIX_Z270H_GAMING_Motherboard (1) because it contains this word: statement

[2021-12-02 16:56:06,246] [DEBUG] [paperless.matching] DocumentType Manual matched on document 1971-06-01 E12509_STRIX_Z270H_GAMING_Motherboard (1) because it contains this word: Manual

[2021-12-02 16:56:06,254] [INFO] [paperless.handlers] Detected 2 potential document types, so we've opted for Statement

[2021-12-02 16:56:06,258] [INFO] [paperless.handlers] Assigning document type Statement to 1971-06-01 E12509_STRIX_Z270H_GAMING_Motherboard (1)

[2021-12-02 16:56:07,644] [DEBUG] [paperless.consumer] Deleting file /usr/src/paperless/src/../consume/E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf

[2021-12-02 16:56:07,657] [ERROR] [paperless.consumer] The following error occured while consuming E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf: [Errno 2] No such file or directory: '/usr/src/paperless/src/../consume/E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf'

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/consumer.py", line 334, in try_consume_file

    os.unlink(self.path)

FileNotFoundError: [Errno 2] No such file or directory: '/usr/src/paperless/src/../consume/E12509_STRIX_Z270H_GAMING_Motherboard (1).pdf'

[2021-12-02 16:56:07,663] [DEBUG] [paperless.parsing.tesseract] Deleting directory /tmp/paperless/paperless-hnz68_xx

Sanity Check output:
All of these orphaned files are left over from other "failed" consumptions of PDF's manuals I downloaded from the internet.

[2021-12-02 17:08:23,307] [ERROR] [paperless.sanity_checker] Thumbnail of document 122 does not exist.
[2021-12-02 17:08:23,314] [ERROR] [paperless.sanity_checker] Checksum mismatch of archived document 122. Stored: f265de68504d60744481090b64f5495c, actual: 1a89fd73c127c139b14d34cf8cf4e84d.
[2021-12-02 17:08:23,319] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/originals/2021/GoControl Zwave Garage GD00Z-8 GC_none_none_none.pdf
[2021-12-02 17:08:23,324] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/originals/2021/MPK mini - User Guide - v1.0_none_Manual_none.pdf
[2021-12-02 17:08:23,329] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/originals/2021/VT20_HappyLight_Liberty_none_Manual_none.pdf
[2021-12-02 17:08:23,334] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/originals/2018/Neato Botvac D5_D3 Connected Wifi-enabled robot vacuum_none_Manual_none.pdf
[2021-12-02 17:08:23,340] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/originals/2017/E12509_STRIX_Z270H_GAMING_Motherboard_none_Statement_none.pdf
[2021-12-02 17:08:23,346] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/originals/2013/RT-AC66U Router_none_Lease_none.pdf
[2021-12-02 17:08:23,358] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/originals/1971/E12509_STRIX_Z270H_GAMING_Motherboard (1)_none_Statement_none.pdf
[2021-12-02 17:08:23,363] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/thumbnails/0000156.png
[2021-12-02 17:08:23,368] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/thumbnails/0000158.png
[2021-12-02 17:08:23,373] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/thumbnails/0000161.png
[2021-12-02 17:08:23,377] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/thumbnails/0000162.png
[2021-12-02 17:08:23,386] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/thumbnails/0000163.png
[2021-12-02 17:08:23,390] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/thumbnails/0000165.png
[2021-12-02 17:08:23,394] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/thumbnails/0000167.png
[2021-12-02 17:08:23,399] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/archive/2021/GoControl Zwave Garage GD00Z-8 GC_none_none_none.pdf
[2021-12-02 17:08:23,403] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/archive/2021/MPK mini - User Guide - v1.0_none_Manual_none.pdf
[2021-12-02 17:08:23,408] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/archive/2021/VT20_HappyLight_Liberty_none_Manual_none.pdf
[2021-12-02 17:08:23,413] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/archive/2018/Neato Botvac D5_D3 Connected Wifi-enabled robot vacuum_none_Manual_none.pdf
[2021-12-02 17:08:23,418] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/archive/2013/RT-AC66U Router_none_Lease_none.pdf
[2021-12-02 17:08:23,422] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/archive/1971/E12509_STRIX_Z270H_GAMING_Motherboard (1)_none_Statement_none.pdf

E12509_STRIX_Z270H_GAMING_Motherboard.pdf
Z-wave__Door_Window_Sensor_DW-ZWAVE2.5-ECO_NOV16.pdf