Memory usage stays high after recognition in docling serve – is this expected?

[inbfor]

For some reason, memory usage doesn't go back after document recognition - because of this we have OOM issue. Is this expected and if not, what can we do? As an OCR engine we use EasyOCR.

[dosubot[bot]]

Hi @inbfor! I'm Dosu and I’m helping the docling team.

High memory usage after recognition in docling-serve with EasyOCR is not expected and is usually a sign that memory cleanup isn't happening after each job. The code does not currently perform explicit memory cleanup—there are no calls to torch.cuda.empty_cache(), gc.collect(), or deletion of large objects after OCR tasks, which can lead to OOM issues, especially under heavy load or repeated use.

To fix this, add GPU memory cleanup in the finally block of the AsyncLocalWorker.loop() method (in docling_serve/engines/async_local/worker.py). For example:

import torch

# ... inside the finally block:
torch.cuda.empty_cache()

If you create job-specific models or tensors, explicitly delete them with del before calling torch.cuda.empty_cache() to ensure memory is released after every task. This approach is confirmed to resolve persistent memory usage and OOM problems in similar setups [source].

For large documents, consider processing in smaller chunks (using page_range or splitting files) and use default image scale unless higher accuracy is needed to further reduce memory footprint [source].

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[dolfim-ibm]

Are you comparing with the CLI usage of Docling? Can you please post which version of Docling you are comparing and which version of Docling Serve?

The last releases (of yesterday) just fixed a memory issue in the docling-parse-v4 which is default in Docling Serve but not in the Docling CLI.

[inbfor]

No, we are not comparing with CLI usage od Docling - we use docling-serve with OpenWebUI.