Skip to content

[Bug]: AI reboot between documents and no VRAM release after finishΒ #863

@AMDFreak2006

Description

@AMDFreak2006

πŸ” Bug Summary

AI gets restarted for each document and VRAM is not released after processing the last document

πŸ“– Description

At first thank you very much for this awesome program.

I use Paperless AI together with Ollama. As soon as Paperless AI analyzes a document, the AI model is loaded into the GPU VRAM and gets processed. Once this is complete and Paperless AI wants to analyze the next document, the AI model is deleted from the VRAM and reloaded (source: Memory usage of the GPU in Windows Task manager). It's basically like rebooting the AI model for each document. Is this intentional? Maybe it is for a clean startup and no AI history mixup of different documents?

In addition, once PaperlessAI has processed the last document in the queue, it does not release the VRAM. The AI model remains open in the VRAM for ever. So what's missing here is the β€œI'm done. Close the AI model again” command to Ollama from PaperlessAI, which seems to be executed between documents but not after the last document was processed.

πŸ”„ Steps to Reproduce

Look at the GPU memory usage during using PaperlessAI and Ollama.
Analyze at least 2 documents automatically in a queue. For example by having 2 documents in PaperlessNGX with a tag what PaperlessAi should analyze (make sure they are not in history section of PaperlessAI of course).
First document: AI Model gets loaded into VRAM.
Second document: VRAM gets released. AI model gets loaded into VRAM again.
Second document finished PaperlessAI processing -> VRAM is never released (until you start analyzing a document again...)

βœ… Expected Behavior

AI model is unloaded and/or VRAM is released after finishing the processing of the last document in the queue.
And in case reloading the AI model after each document is unwanted, this shouldn't happen.

❌ Actual Behavior

VRAM is not released after last document.
AI model is reloaded at start of each document analysis.

🏷️ Paperless-AI Version

3.0.9

πŸ“œ Docker Logs

not applicable

πŸ“œ Paperless-ngx Logs

πŸ–ΌοΈ Screenshots of your settings page

No response

πŸ–₯️ Desktop Environment

Windows

πŸ’» OS Version

Windows 11

🌐 Browser

None

πŸ”’ Browser Version

No response

🌐 Mobile Browser

No response

πŸ“ Additional Information

  • I have checked existing issues and this is not a duplicate
  • I have tried debugging this issue on my own
  • I can provide a fix and submit a PR
  • I am sure that this problem is affecting everyone, not only me
  • I have provided all required information above

πŸ“Œ Extra Notes

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions