Extract first page text from Django InMemoryUploadedFile

[mnoah66]

My script works perfectly fine when running from the command line and using fitz.open('my_local_file.pdf')

The problem is when trying to use the same script in Django using an uploaded file (InMemoryUploadedFile).

views.py

def upload_file(request):
    ...
    uploaded_file = request.FILES['frontend_file']
    d,n,l = pdf_functions.get_details_from_pcpt(uploaded_file)

pdf_functions.py

def get_details_from_pcpt(uploaded_file):
    doc = fitz.open(uploaded_file)
    page = doc[0]  
    words = page.get_text("words")
    print(f"{words=}")

    n = fitz.Rect(250, 120, 520,  145) 
    d = fitz.Rect(195, 144, 520,  195)
    l = fitz.Rect(541, 60, 600,  80)

    ddd_id = page.get_textbox(d)
    name = page.get_textbox(n)
    last_updated = page.get_textbox(l)
    
    return ddd_id, name, last_updated

Results in:

FileNotFoundError at /core/upload/
no such file: 'my_local_file.pdf'

Changing this line:
doc = fitz.open(uploaded_file)
To:
doc = fitz.open(stream=uploaded_file.read(), filetype="pdf")

Results in an empty list when trying to print out the words: words=[]

Pretty new to this and feel like I am missing something really obvious. Any help would be appreciated.

[JorjMcKie]

PyMuPDF cannot read from the internet - which is presumably what you are trying to do here.
The term stream I am using in PyMuPDF does not refer to internet access - which may be confusing at times.

So uploaded_file does not contain anything meaningful probably.
Although I am confused about why no exception was raised: this should happen in cases of a zero-length file ...

To try out the from-memory case do this with any valid document:

inpdf = open("some.pdf", "rb")
pdfbytes = inpdf.read()
inpdf.close()
doc=fitz.open("pdf", pdfbytes)

[mnoah66]

Hmm, so upon further experimenting, doc = fitz.open(stream=uploaded_file.read(), filetype="pdf") is working except for when trying to access the first page.

page0 = doc[0]
page1 = doc[1]   

page_0_words = page0.get_text("words") # Results in empty list []
page_1_words = page1.get_text("words") # Results in expected list of words and their locations:
 [
    (26.20800018310547, 132.35606384277344, 94.00122833251953, 146.1246795654297, 'Achievements', 0, 0, 0),
    (52.41600036621094, 158.00509643554688, 66.9223861694336, 170.36972045898438, 'Jon', 1, 0, 0),
    (69.42411041259766, 158.00509643554688, 96.43910217285156, 170.36972045898438, 'stated,', 1, 0, 1),
    (98.94082641601562, 158.00509643554688, 131.1392364501953, 170.36972045898438, '"moving', 1, 0, 2),
    ...
]

I've confirmed that there is indeed text in the first page of the PDF, and it is accessible by PyMuPDF through the regular fitz.open('my_local_pdf.pdf')

Is it possible stream is ignoring the first page?

[JorjMcKie]

Is it possible stream is ignoring the first page?

No. But possibly the first page contains no text - only images or line art.

[JorjMcKie]

to test, store the memory pdf again as a PDF or make a pixmap of that first page and store that as a PNG.

[mnoah66]

No. But possibly the first page contains no text - only images or line art.

The first page definitely has text.

>doc = fitz.open("my_local_pdf.pdf")
>print(doc[0].get_text("words"))
> [
(145.29600524902344, 255.85806274414062, 159.1846160888672, 269.5967102050781, 'SC', 0, 0, 0),
(145.58399963378906, 272.92205810546875, 166.14193725585938, 286.66070556640625, 'SCS', 1, 0, 0),
(145.29600524902344, 304.89007568359375, 151.96533203125, 318.62872314453125, 'E', 2, 0, 0),
(145.58399963378906, 288.90606689453125, 152.25332641601562, 302.64471435546875, 'P', 3, 0, 0),...
]

I will try some more PDFs.

[mnoah66]

@JorjMcKie looks like I got it working. I had to put uploaded_file.seek(0) before fitz.open with the stream option. With this change the first page of the PDF became available.

Here is my code now:

def get_details_from_pcpt(uploaded_file):
    uploaded_file.seek(0)
    doc = fitz.open(stream=uploaded_file.read(), filetype="pdf")
    page = doc[0]  
    words = page.get_text("words")
    print(f"{words=}") # words=[
    (145.29600524902344, 255.85806274414062, 159.1846160888672, 269.5967102050781, 'SC', 0, 0, 0),
    (145.58399963378906, 272.92205810546875, 166.14193725585938, 286.66070556640625, 'SCS', 1, 0, 0),
    (145.29600524902344, 304.89007568359375, 151.96533203125, 318.62872314453125, 'E', 2, 0, 0),
    (145.58399963378906, 288.90606689453125, 152.25332641601562, 302.64471435546875, 'P', 3, 0, 0),
    ...]

[mnoah66]

In case anyone else sees this and misses the thread above. I had to put uploaded_file.seek(0) prior to trying to read the file in fitz.open(stream=uploaded_file.read(), filetype="pdf")

My final function looks like this:

def get_details_from_pcpt(uploaded_file):
    uploaded_file.seek(0)
    doc = fitz.open(stream=uploaded_file.read(), filetype="pdf")
    page = doc[0]  
    words = page.get_text("words")
    print(f"{words=}") # words=[
    (145.29600524902344, 255.85806274414062, 159.1846160888672, 269.5967102050781, 'SC', 0, 0, 0),
    (145.58399963378906, 272.92205810546875, 166.14193725585938, 286.66070556640625, 'SCS', 1, 0, 0),
    (145.29600524902344, 304.89007568359375, 151.96533203125, 318.62872314453125, 'E', 2, 0, 0),
    (145.58399963378906, 288.90606689453125, 152.25332641601562, 302.64471435546875, 'P', 3, 0, 0),
    ...]

Not sure how/why the pointer/cursor moves when trying to read the file. Perhaps because I have opened the file already in another part of views.py before sending it off to this function, resulting in an issue of trying to read the file twice: see here.