formula enrich cause memory spikes

[liqing-create]

I was trying to use formula-enrich option to capture equations properly. however, it usually cause a memory spike to 99%, making the entire process stall. I was using docling "example.pdf" --from pdf --to md --table-mode accurate --enrich-formula --pdf-backend pypdfium2 --verbose --device cpu. if --enrich-formula was not added, it works smoothly, takes 30 - 40% memory. but once --enrich-formula is added, it starts at 40% but it will suddenly spike to 99%. what type of hardware do I need for this feature to work properly? I have a RAM of 16G. Thank you in advance.

[dolfim-ibm]

We have some fix which reduces the batch size and should allow to control the overall memory used. See #878

[liqing-create]

Thank you very much for your response. The memory issue is greatly alleviated (although I still have some 99% moments, but it will come down unlike before it was just roller coaster). I think the current version of docling is much slower than before (I started to use docling in December last year). For the same pdf with the same features selected, if it took around 80 sec to finish back in December, now it needs 240 sec. I tried to use GPU for accelerations. For the most part, it works great, for the same paper, it might take 45 sec. However, I encountered errors related to this custom kernel file. In particular, I saw compilation errors along the lines of: error: no suitable conversion function from "const at::DeprecatedTypeProperties" to "c10::ScalarType" exists. This error appears to be triggered by the use of value.type() in the custom kernel code. I attempted to patch the code (e.g., replacing value.type() with value.scalar_type()) in ms_deform_attn_cuda.cu within the site-packages of my virtual environment, but docling’s installation process uninstalls and reinstalls dependencies, any direct patch I make in site‑packages might be overwritten. Do you have any suggestion on how to solve this kernel issue? Thank you in advance!

OS: Windows 11 Home
Python: 3.11.5
docling Version: 2.18.0 (installed from GitHub, pip install --no-cache-dir git+https://github.com/DS4SD/docling.git)
Transformers Version: 4.48.2 (as installed as a dependency)
CUDA: 12.8 (nvcc shows CUDA 12.8, and PyTorch is installed with CUDA support)
Virtual Environment: Created with python -m venv (isolated environment)