Feature Request: Enable Agent Mode Overrides & Improve Developer Guidance for Local Models

[BarrettBytes]

Validations

• I believe this is a way to improve. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that requests the same enhancement

Problem

Agent mode is currently hard-disabled for models without tool support metadata, which blocks usage of private or fine-tuned models that could work via prompt engineering. There should be an override to force agent mode, along with docs showing the expected function call format so users can experiment.

Solution

Issue:

When trying to use Gemma-3, I received the following error:

registry.ollama.ai/library/gemma3:latest does not support tools

This hardcoded block is overly restrictive. Many models—even those without built-in tool use—can be made to work using prompt engineering, if we just know the format the model is expected to output.

Worse, if a user is running a privately fine-tuned model, there may be no metadata available at all for Continue to detect capabilities. In these cases, there’s no good reason to block agent mode outright. The current behavior actively prevents experimentation and development with custom LLMs.

---

Feature Suggestions:

1. Allow Manual Agent Mode Override:
   Users should have a way to force enable agent mode—even if the system doesn’t recognize tool support. Add a flag or config setting to allow it, with a warning if needed.

2. Document Tool Call Format & Prompts:
   Please add documentation that includes:

   • The required format for function/tool call outputs.
   • Example prompts that trigger tool use.
   • A breakdown of the system prompts that manage agent behavior.

   This will allow developers to adapt their prompts for compatibility, especially when using nonstandard or private models.

3. Improve Error Messages:
   Instead of just saying the model doesn’t support tools, explain:

   • What the system expects from the model.
   • What capabilities are missing or undetected.
   • How the user can experiment or override the limitation.

---

Why This Matters:

Open-source tooling should empower developers, not gatekeep based on assumptions. Just because a model doesn’t advertise tool support doesn’t mean it can’t work. With the right prompt engineering, many models—including private fine-tunes—can handle tool calling just fine. The only thing in the way is an artificial, enforced limitation.

Thanks for building an awesome tool—let’s make it even more hackable and developer-friendly.

[Pyroboomka]

You can add models[0].capabilities.tool_use = true in config, is that not enough?
https://docs.continue.dev/reference#models:~:text=not%20currently%20used.-,capabilities,-%3A%20Array%20of%20strings

[BarrettBytes]

No if you add this to the config and then try use a local model through llama.cpp the options for agent will appear and you will be able to select agent but it will not work, it will not be given any of the tools, if you ask the model about the chat history it will tell you it knows nothing about any tools and was never told any tools exist. Because even though this makes agent appear as an option the actual tools are never provided to the model. I have tested with a crappy model like gemini flash and when asked about its chat history it will list all the tools but if you use any local model it will swear it has never heard of any tools and they don't exist, even the suggested models for ollama will work for ollama but not llama.cpp.

note this model works fine in https://github.com/acoliver/llxprt-code and is able to use tool calls fine - it's a bit stupid and not useful but it still successfully calls the tools there - so it's definitely something wrong with this project. See here is the model working fine in llxprt

so its's 100% a continue.dev issue that should be fixed

[bdougie]

This seems relevant to the demo we saw from Chad @TyDunn