-
-
Notifications
You must be signed in to change notification settings - Fork 260
Description
π Bug Summary
AI gets restarted for each document and VRAM is not released after processing the last document
π Description
At first thank you very much for this awesome program.
I use Paperless AI together with Ollama. As soon as Paperless AI analyzes a document, the AI model is loaded into the GPU VRAM and gets processed. Once this is complete and Paperless AI wants to analyze the next document, the AI model is deleted from the VRAM and reloaded (source: Memory usage of the GPU in Windows Task manager). It's basically like rebooting the AI model for each document. Is this intentional? Maybe it is for a clean startup and no AI history mixup of different documents?
In addition, once PaperlessAI has processed the last document in the queue, it does not release the VRAM. The AI model remains open in the VRAM for ever. So what's missing here is the βI'm done. Close the AI model againβ command to Ollama from PaperlessAI, which seems to be executed between documents but not after the last document was processed.
π Steps to Reproduce
Look at the GPU memory usage during using PaperlessAI and Ollama.
Analyze at least 2 documents automatically in a queue. For example by having 2 documents in PaperlessNGX with a tag what PaperlessAi should analyze (make sure they are not in history section of PaperlessAI of course).
First document: AI Model gets loaded into VRAM.
Second document: VRAM gets released. AI model gets loaded into VRAM again.
Second document finished PaperlessAI processing -> VRAM is never released (until you start analyzing a document again...)
β Expected Behavior
AI model is unloaded and/or VRAM is released after finishing the processing of the last document in the queue.
And in case reloading the AI model after each document is unwanted, this shouldn't happen.
β Actual Behavior
VRAM is not released after last document.
AI model is reloaded at start of each document analysis.
π·οΈ Paperless-AI Version
3.0.9
π Docker Logs
not applicableπ Paperless-ngx Logs
πΌοΈ Screenshots of your settings page
No response
π₯οΈ Desktop Environment
Windows
π» OS Version
Windows 11
π Browser
None
π’ Browser Version
No response
π Mobile Browser
No response
π Additional Information
- I have checked existing issues and this is not a duplicate
- I have tried debugging this issue on my own
- I can provide a fix and submit a PR
- I am sure that this problem is affecting everyone, not only me
- I have provided all required information above
π Extra Notes
No response