[Bug]: Docker API LLM Operations / Extractions

[Codinator007]

crawl4ai version

0.7.6

Expected Behavior

Same as other people I am not able to get llm extraction to run.
I have setup via docker compose and want to use for llm extraction my local (same vm) vllm or ollama endpoints.
I wish I could turn on LiteLLM Debug Logs because right now i only see this in the logs:

stage_crawl4ai-1  | LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
stage_crawl4ai-1  | 
stage_crawl4ai-1  | 
stage_crawl4ai-1  | Provider List: https://docs.litellm.ai/docs/providers

I tried nearly everything:

• config.yml with llm config
• .llm.env with custom LLM_BASE_URL
• as Provider i tried
• • LLM_PROVIDER=ollama/gpt-oss-20b
• • LLM_PROVIDER=hosted_vllm/gpt-oss-20b
• • LLM_PROVIDER=hosted_vllm/openai/gpt-oss-20b
• • etc.

Current Behavior

no debug logs of litellm
extraction with llm not work.
for example i use /md endpoint with f:llm & q:Extract only x

Is this reproducible?

Yes

Inputs Causing the Bug

Steps to Reproduce

Code snippets

OS

docker ubuntu 22.04

Python version

3.10.12 on the host

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

No response

[SohamKukreti]

Have you tried running it with a hosted LLM?

[mratsim]

Have you tried running it with a hosted LLM?

ollama and hosted_vllm are self-hosted

[arinhouck]

I was having the same issue when trying to access my llama cpp server on localhost:8080 and using any model (specifically just trying gpt 20b oss). Then I recalled that docker doesn't have access to localhost.

I changed to host.docker.internal:8080 instead and it started working. If you are trying to access localhost I'd assume your issue might be similar.

[SohamKukreti]

@Codinator007 Can you try what @arinhouck suggested. Let me know if it works, I'll close the issue then.
Thanks!

[Codinator007]

Sorry guys for the late reply. I was on holiday.
Since Ollama is in the same docker compose as crawl4ai I use http://ollama:11434 as base url. I also tried /v1 and ollama-chat/gpt-oss:20b as provider.
Right now my best approach was this curl JSON:

{
  "urls": [
    "<shop_url>"
  ],
  "crawler_config": {
    "type": "CrawlerRunConfig",
    "params": {
      "magic": true,
      "override_navigator": true,
      "verbose": true,
      "extraction_strategy": {
        "type": "LLMExtractionStrategy",
        "params": {
          "instruction": "Extract the netto price of this article page. Return JSON only.",
          "extraction_type": "schema",
          "schema": {
            "type": "object",
            "properties": {
              "article_nr": {
                "type": "string"
              },
              "price": {
                "type": "string",
                "description": "Netto price (net), include currency if present"
              }
            },
            "required": [
              "price"
            ],
            "additionalProperties": false
          },
          "llm_config": {
            "type": "LLMConfig",
            "params": {
              "provider": "ollama_chat/gpt-oss:20b",
              "base_url": "http://prod_ollama:11434",
              "temperature": 0.2
            }
          }
        }
      }
    }
  }
}

Now I dont get an LiteLLM error what is great but I still dont get an response but ollama gets called regarding the logs:

prod_crawl4ai-1    | 07:53:56 - LiteLLM:INFO: utils.py:3440 - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 2025-12-16 07:53:56,834 - LiteLLM - INFO - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 07:53:56 - LiteLLM:INFO: utils.py:3440 - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 2025-12-16 07:53:56,835 - LiteLLM - INFO - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 07:53:56 - LiteLLM:INFO: utils.py:3440 - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 2025-12-16 07:53:56,835 - LiteLLM - INFO - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 07:53:56 - LiteLLM:INFO: utils.py:3440 - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 2025-12-16 07:53:56,836 - LiteLLM - INFO - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 07:53:56 - LiteLLM:INFO: utils.py:3440 - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 2025-12-16 07:53:56,837 - LiteLLM - INFO - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 07:53:56 - LiteLLM:INFO: utils.py:3440 - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 2025-12-16 07:53:56,837 - LiteLLM - INFO - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 07:53:56 - LiteLLM:INFO: utils.py:3440 - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 2025-12-16 07:53:56,838 - LiteLLM - INFO - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 07:53:56 - LiteLLM:INFO: utils.py:3440 - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 2025-12-16 07:53:56,838 - LiteLLM - INFO - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 07:53:56 - LiteLLM:INFO: utils.py:3440 - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 2025-12-16 07:53:56,838 - LiteLLM - INFO - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 07:53:56 - LiteLLM:INFO: utils.py:3440 - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 2025-12-16 07:53:56,839 - LiteLLM - INFO - 
prod_crawl4ai-1    | LiteLLM completion() model= gpt-oss:20b; provider = ollama_chat
prod_crawl4ai-1    | 7:M 16 Dec 2025 07:54:24.025 * 1 changes in 3600 seconds. Saving...
prod_crawl4ai-1    | 7:M 16 Dec 2025 07:54:24.025 * Background saving started by pid 9219
prod_crawl4ai-1    | 9219:C 16 Dec 2025 07:54:24.026 * DB saved on disk
prod_crawl4ai-1    | 9219:C 16 Dec 2025 07:54:24.026 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
prod_crawl4ai-1    | 7:M 16 Dec 2025 07:54:24.125 * Background saving terminated with success
prod_ollama-1      | [GIN] 2025/12/16 - 07:57:55 | 200 |         7m52s |      172.19.0.8 | POST     "/api/chat"
prod_ollama-1      | time=2025-12-16T07:57:55.842Z level=WARN source=runner.go:186 msg="truncating input prompt" limit=4096 prompt=7987 keep=4 new=4096
prod_ollama-1      | [GIN] 2025/12/16 - 07:58:31 | 200 |         8m28s |      172.19.0.8 | POST     "/api/chat"
prod_ollama-1      | time=2025-12-16T07:58:31.429Z level=WARN source=runner.go:186 msg="truncating input prompt" limit=4096 prompt=8467 keep=4 new=4096

[Codinator007]

not closed... sorry

as result i get

Error code:
ECONNABORTED

Full message:
timeout of 300000ms exceeded

[Ahmed-Tawfik94]

Root Cause Analysis

• Primary cause: LLM calls to local Ollama take several minutes; /md is synchronous and exceeds typical 300s client timeouts.
• Contributing factors:
  • LiteLLM default timeout (~600s) eventually terminates the server-side request.
  • /md does not expose LLM errors and can return empty markdown or hang until client timeout.
  • Long synchronous calls can block other API requests when worker count is low.
• Evidence:
  • /md timed out at 300s on docs.crawl4ai.com with ollama/llama3.2:1b.
  • /llm/job finished with a LiteLLM timeout at ~600s.
  • Logs show provider/model selected correctly.

Resolution Check (Current Version)

• Status: not resolved
• Evidence:
  • /md remains synchronous without timeout controls or job fallback.
  • No Docker API toggle for LiteLLM debug logging.

Suggested Fixes (Backward-Compatible)

1. Add configurable LLM request timeout (e.g., env LLM_REQUEST_TIMEOUT_S) passed into LiteLLM via extra_args.
2. Add a job-based /md variant or automatic job fallback when LLM calls exceed a short threshold.
3. Surface LLM errors in /md responses instead of returning empty markdown.
4. Add an env toggle to enable LiteLLM debug logging via litellm._turn_on_debug().