[Enhancement]: User Tool on/off granularity for Agents

[leondape]

What features would you like to see added?

I would like to have Agent Tools/Capabilities available AND toggleable in the Chat Window like in normal Chats.
Right now, they are hidden away when using an Agent.

Also, during Agent creation, it should be possible to set the default tool state (attached/not attached, on/off). When starting a new chat with an agent the default state shall apply.

More details

Problem

Currently, when chatting with an Agent, the settings for Tool usage (Tools, Web, Code, MCPs) are hidden away. This UI decision makes sense, if you assume that the "choice" of Tool selection was made during agent creation + prompt engineering and that the choice at creation time is optimal.
The downside however is, that when creating more advanced agents with certain behavior and a narrowed selection of tools, that the agents get slow and expensive.
I tried to work around this by using small/cheap/fast/stupid models for the agent, which instructs users to switch to more capable models if results are bad or the requirements are complex. This is an unnecessary limitation if we implement attaching tools to agents but disabling them.

Suggestion 1

What I am basically suggesting is a chatgpt-like approach:

• Build your own agents with custom instructions
• Setup Tools they can use:
  • Give them only tools they are supposed / allowed to use
  • Determine wether tools should be enabled from the start: eg. enable Office-MCP-Server always because it is a frequent requirement for the agent but don't enable Image-Generation with a lot of tools or Web-search.
• through Prompting we can make the agent aware that it can use more tools but ask the user to enable it when required.
• User now uses "General-Office-Agent": User can access their outlook calendar but before generating an image they need to enable the tool.

This whole flow would make us more flexible in choosing models, maintain high speed and lower token consumption.

Alternative

With the current capabilities one could achieve similar behavior by using modelspecs and adding system prompts to models.
BUT: This way we cannot limit visible MCP Servers the user can attach. Normal Users could be overwhelmed when MCP Options are extensive. Also Only MPCs can be attached, Tools can not as they require Agents.

=> Therefore I would prefer Suggestion 1 and extending the Agent tools selections and Chatbox UI Elements for Agents.

Which components are impacted by your request?

No response

Pictures

• Agent Schema
• Chatbox UI Element
• Caching of Tool Preferences (or just not caching them per user and always applying the defaults)

Code of Conduct

• I agree to follow this project's Code of Conduct