requirements.txt needs ”pycryptodome” for venv

[notanaha]

Please provide us with the following information:

This issue is for a: (mark with an x)

- [x ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Run "./scripts/prepdocs.ps1" after modifying resource keys

Any log messages given by the failure

Running "prepdocs.py"
Ensuring search index gptkbindex exists
Search index gptkbindex already exists
Processing files...
Processing 'C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo/data\taxbook.pdf'
Traceback (most recent call last):
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts\prepdocs.py", line 312, in
upload_blobs(filename)
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts\prepdocs.py", line 72, in upload_blobs
for i in range(len(pages)):
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf_page.py", line 2326, in len
return self.length_function()
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf_reader.py", line 449, in _get_num_pages
return self.trailer[TK.ROOT]["/Pages"]["/Count"] # type: ignore
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf\generic_data_structures.py", line 269, in getitem
return dict.getitem(self, key).get_object()
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf\generic_base.py", line 281, in get_object
obj = self.pdf.get_object(self)
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf_reader.py", line 1296, in get_object
retval = self._get_object_from_stream(indirect_reference) # type: ignore
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf_reader.py", line 1219, in _get_object_from_stream
obj_stm: EncodedStreamObject = IndirectObject(stmnum, 0, self).get_object() # type: ignore
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf\generic_base.py", line 281, in get_object
obj = self.pdf.get_object(self)
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf_reader.py", line 1360, in get_object
retval = self._encryption.decrypt_object(
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf_encryption.py", line 974, in decrypt_object
return cf.decrypt_object(obj)
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf_encryption.py", line 187, in decrypt_object
obj._data = self.stmCrypt.decrypt(obj._data)
File "C:\Users\notanaha\Documents\GitHub\azure-search-openai-demo\scripts.venv\lib\site-packages\pypdf_encryption.py", line 149, in decrypt
raise DependencyError("PyCryptodome is required for AES algorithm")
pypdf.errors.DependencyError: PyCryptodome is required for AES algorithm

Expected/desired behavior

./scripts/prepdocs.ps1 completes successfully

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)
Windows 11

Versions

Python 3.10.10

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

[jhigh89]

Hey! This most likely has something to do with a particular pdf(s) that you are trying to read and chunk in that step. Just go to the requirements.txt and add PyCryptodome (the latest at the time of me writing this is pycryptodome 3.17). When you run the ./scripts/prepdocs.sh next time, it should complete without issues. Obviously, only add the library to the requirements if you trust it.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.

[pamelafox]

Thanks for the report back when the project still relied on an earlier ingestion path. Since then the ingestion pipeline was refactored: the default (and recommended) document ingestion now uses Azure AI Document Intelligence (DocumentAnalysisParser in prepdocslib/pdfparser.py) which handles many formats without depending on PyCryptodome. The fallback / optional "Local PDF Parsing" path (enabled only when the Local PDF Parsing option is explicitly selected) uses pypdf for basic text extraction; that path also hasn’t required adding PyCryptodome in typical scenarios with unencrypted PDFs.

Given this architectural shift and lack of current reports about encrypted PDF failures in the maintained flow, the original request to add PyCryptodome to requirements.txt is now obsolete. We’ll close this as outdated. If you still encounter issues processing encrypted PDFs when explicitly using Local PDF Parsing, please open a new issue with details (and, if possible, a reproducible sample or encryption characteristics). Thank you for helping improve the project!