Roo Code disconnects before LM Studio can finish its response

[dabockster]

App Version

3.25.4

API Provider

LM Studio

Model Used

Deepseek R1 14b/32b, Qwen2.5 Coder Instruct 32b - all GGUF from unsloth and bartowski on HuggingFace

Roo Code Task Links (Optional)

No response

🔁 Steps to Reproduce

1. Connect Roo Code to a sufficiently large model on LM Studio where processing needs to be split between the GPU and CPU.
2. Wait for LM Studio to process the input and relay the output.
3. Roo Code will time out after 1-2 minutes before LM Studio can fully process my input - sometimes making it drop its context and start processing from the beginning.

💥 Outcome Summary

Expected Roo Code to wait for LM Studio to compute its response instead of timing out, regardless of how long that amount of time might be. I care about accuracy and its ability to execute using what's available on my local system, and not execution speed at all.

When it has ran, though, it has been excellent! I just can't have it timing out and retrying 4+ times before LM Studio has enough context built up. It feels like that timeout was built for a cloud service and not a local program.

📄 Relevant Logs or Errors (Optional)

[dabockster]

That's an awesome AI you guys have! Can you make that change apply to all local providers like Ollama and others as well, and not just for LM Studio?

[daniel-lxs]

Hey, I took a look at this timeout issue with LM Studio and other local providers. Here's what I found:

The problem is that the OpenAI SDK clients used by LM Studio, Ollama, and other OpenAI-compatible providers don't have any timeout configuration. When you're running large models that need to split processing between GPU and CPU, the default timeout (which seems to be around 60-120 seconds based on your description) kicks in before LM Studio can finish processing.

Looking at the code, I can see that when these providers create their OpenAI clients, they don't pass any timeout options:

• LM Studio: Creates client without timeout at src/api/providers/lm-studio.ts:25-28
• Ollama: Creates client without timeout at src/api/providers/ollama.ts:26-28
• OpenAI/Compatible: Creates client without timeout at src/api/providers/openai.ts:67-72

The OpenAI SDK does support a timeout option in the client constructor, but it's not being used.

Proposed Solution

Add a VSCode setting that allows users to configure the timeout for all API providers. This would be especially useful for local providers where response times can vary significantly based on hardware capabilities and model size.

Implementation Details:

1. Add a new VSCode setting in src/package.json:

"roo-cline.apiRequestTimeout": {
  "type": "number",
  "default": 600,
  "minimum": 0,
  "maximum": 3600,
  "description": "Maximum time in seconds to wait for API responses (0 = no timeout, 1-600s for cloud providers, higher for local providers)"
}

2. Update each provider to read this setting and pass it to the OpenAI client:

const timeoutSeconds = vscode.workspace
  .getConfiguration("roo-cline")
  .get<number>("apiRequestTimeout", 600);

this.client = new OpenAI({
  baseURL: ...,
  apiKey: ...,
  timeout: timeoutSeconds * 1000, // Convert to milliseconds
});

3. Apply to all affected providers:
   • LM Studio handler
   • Ollama handler
   • OpenAI handler (which covers OpenAI-compatible providers)
   • Any other providers using the OpenAI SDK

This would give users full control over the timeout, allowing them to set higher values for local providers that need more processing time. The default of 600 seconds (10 minutes) should be sufficient for most local model scenarios while still providing reasonable limits for cloud providers.