How to download and run models locally

[jereesh]

I am trying to use Docling , It fails in standard mode since it tries to download huggingface models and since I am within corporate proxy there are restrictions in programmatic access, I tried to download individual files from hugging face in c:\users\xxx.cache\huggingface as well as in c:\users\xxx.cache\docling , and configured and env variable DOCLING_MODELS_PATH but still it is trying to download from huggingface ,
Have tried with vision models with Ollama. it is working but the result is not good, I have used gabegoodheart/granite-docling:258m model for this . How do I manually download individual files for a model from hugging face and configure it so that it won't try to connect to hugging face

[dosubot[bot]]

To run Docling completely offline and avoid any downloads from Hugging Face, you need to manually download all required model files and configure Docling to use them via the DOCLING_SERVE_ARTIFACTS_PATH environment variable (not DOCLING_MODELS_PATH, which is not recognized by Docling) reference.

Steps:

1. Download model files:
   If you have internet access outside your proxy, use:

   docling-tools models download --all -o <your_local_models_dir>
   

   If you can't run this command, manually download the required files (like model.safetensors, config.json, preprocessor_config.json) from the Hugging Face repo using your browser and place them in a directory. The directory structure should match what Docling expects—typically, each model in its own subfolder (e.g., ds4sd--docling-models).

2. Set the environment variable:
   Point Docling to your local models directory by setting:

   DOCLING_SERVE_ARTIFACTS_PATH=<absolute_path_to_your_local_models_dir>
   

   This must be set in your environment before starting Docling. For containers, mount the models directory and set the variable inside the container.

3. Check directory structure:
   Make sure all required files are present and not nested too deeply. For example, model.safetensors, config.json, and preprocessor_config.json should be directly inside the relevant model subfolder reference.

Common pitfalls:

• Using DOCLING_MODELS_PATH instead of DOCLING_SERVE_ARTIFACTS_PATH (the latter is correct) reference.
• Incorrect folder names or missing files—Docling will try to download from Hugging Face if it can't find what it needs locally.
• Simply copying files to the Hugging Face cache directory is not enough; explicit configuration is required.

For more details and troubleshooting, see the official Docling documentation and FAQ.

If you follow these steps, Docling should run fully offline and never attempt to connect to Hugging Face.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[jereesh]

what is the directory structure I have a directory called ds2sd--docling-layout-heron

[dosubot[bot]]

For Docling to recognize and use your local ds2sd--docling-layout-heron model (and avoid downloading from Hugging Face), your directory structure should look like this:

<ARTIFACTS_PATH>/
  ds2sd--docling-layout-heron/
    model.safetensors
    config.json
    preprocessor_config.json

Set the environment variable DOCLING_SERVE_ARTIFACTS_PATH to the parent directory (<ARTIFACTS_PATH>) that contains the ds2sd--docling-layout-heron folder. All three files—model.safetensors, config.json, and preprocessor_config.json—must be directly inside the ds2sd--docling-layout-heron folder, not nested further. If any are missing or the structure is different, Docling will try to download the model and fail in a restricted environment reference.

If you manually download the files, make sure the folder name matches the repo ID with / replaced by -- (e.g., ds2sd--docling-layout-heron for ds2sd/docling-layout-heron) reference.

Let me know if you need a diagram or have questions about other models!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[jereesh]

@dosu , I have downloaded the files and set the env variable still it tries to connect to huggingface

[dosubot[bot]]

Docling will still try to connect to Hugging Face if it can't find the required model files in the exact local directory structure it expects, even if the environment variable is set. The most common causes are: the environment variable is not set to the correct path, the directory structure or file names are off (including case sensitivity), or the files are missing or misnamed reference reference.

To troubleshoot, please check:

• The environment variable must be DOCLING_SERVE_ARTIFACTS_PATH (not DOCLING_MODELS_PATH), set to the absolute path of the parent folder containing all your model subfolders.
• Each model (like ds2sd--docling-layout-heron) should be a subfolder inside that path, with model.safetensors, config.json, and preprocessor_config.json directly inside it—not nested further.
• Folder names must match the repo ID with / replaced by -- (e.g., ds4sd--docling-layout-heron for ds4sd/docling-layout-heron).
• config.json must contain a "model_type" key for each model reference.
• The environment variable must be visible to the process running Docling (e.g., set in the same shell/session or inside the container if using Docker).

If you can, share the exact value of DOCLING_SERVE_ARTIFACTS_PATH and a directory listing of your models folder. Also, check any error logs for missing file messages—they usually point to the exact file or folder Docling can't find reference.

Let me know if you need help checking the structure or want to share your setup for a quick review!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}