Error: Cannot copy out of meta tensor; no data! Use torch.nn.Module.to_empty() instead of torch.nn.Module.to()

[Arunsmail94]

Hi,
I'm encountering the following error while using Docling on some files (not all):

"Cannot copy out of meta tensor; no data! Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module from meta to a different device."

The code should process all files without errors, as it does when run locally. The error occurs only on some files when hosted on an AWS EC2 Linux instance.

Environment:
Docling: 2.39.0
Docling-core: 2.42.0
Docling-ibm-models: 3.8.1
Docling-parse: 4.0.1
Python: 3.9.22
OS: Linux/UNIX (AWS EC2)
Device: CPU

This is my code:

Global DocumentConverter

pipeline_options = PdfPipelineOptions()
pipeline_options.images_scale = IMAGE_RESOLUTION_SCALE
pipeline_options.generate_picture_images = True

pipeline_options.accelerator_options = AcceleratorOptions(
num_threads=multiprocessing.cpu_count(),
device=AcceleratorDevice.CPU
)

doc_converter = DocumentConverter(
format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)}

)
conv_res = doc_converter.convert(file_path)

How can I resolve this error? Thanks in advance

[dosubot[bot]]

Hi @Arunsmail94! I'm Dosu and I’m helping the docling team.

This error happens when a PyTorch model is initialized on a "meta" device (a placeholder with no real data) and then moved to a real device using .to(), but the model weights haven't been fully loaded yet. This is more likely in certain async or multi-threaded contexts, or when specific files trigger model loading paths that hit this race condition. The issue is environment-sensitive, which explains why it only happens on AWS EC2 and not locally.

The maintainers fixed this in docling-ibm-models v3.7.0 and docling v2.40.0. Upgrading both packages to at least those versions should resolve the problem. If you can't upgrade, you would need to patch the code to use torch.nn.Module.to_empty() instead of .to() when moving from meta to a real device, or ensure model weights are fully loaded before device transfer, but these workarounds require code changes and aren't built into your current versions. More details and discussion are available in this issue.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[Arunsmail94]

Hi Dosu,
Thank you for the guidance! Upgrading to docling-ibm-models v3.8.1 and docling v2.40.0 resolved the issue. It's now working on the AWS EC2 instance. Appreciate the support!