Skip to content

[Tool Description] Improve Foundry tool description for better LLM selection #826

@charris-msft

Description

@charris-msft

Problem

The Foundry tool description lacks the detail and clarity needed for optimal LLM tool selection, resulting in an estimated confidence score of ~0.2 (well below target of ≥0.5).

Current Description Issues:

  • Missing context about what AI Foundry is
  • Vague purpose without specific service details
  • Missing usage guidance (when to use vs when not to use)
  • Poor parameter explanation
  • Missing limitations/caveats
  • Formatting issues (missing space after period)
  • Too brief for service complexity

Current Description

Foundry service operations - Commands for listing and managing services and resources in AI Foundry.This tool is a hierarchical MCP command router. Sub commands are routed to MCP servers that require specific fields inside the "parameters" object. To invoke a command, set "command" and wrap its args in "parameters". Set "learn=true" to discover available sub commands.

Suggested Improvement

Foundry service operations - Commands for listing and managing Azure AI Foundry services and resources including AI models, endpoints, deployments, and workspace configurations. Use this tool when you need to work with Azure AI Foundry projects, list available AI models, manage model deployments, configure AI endpoints, or query AI workspace resources and settings. This tool is designed for AI and machine learning workflow management within Azure AI Foundry environments. Do not use this tool for general Azure AI services outside of Foundry, Azure Machine Learning studio operations, or Azure OpenAI direct management - this tool specifically targets Azure AI Foundry service operations. This tool is a hierarchical MCP command router where sub-commands are routed to MCP servers that require specific fields inside the "parameters" object. To invoke a command, set "command" and wrap its arguments in "parameters". Set "learn=true" to discover available sub-commands for different AI Foundry operations and resource types.

Acceptance Criteria

  • Update tool description to include AI Foundry context
  • Explain specific AI/ML services and resources covered
  • Clarify when NOT to use this tool
  • Detail the types of operations available
  • Fix formatting issues
  • Achieve confidence score ≥0.5 in tool selection testing

Related: Tool description review for improved LLM selection accuracy

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

Status

Untriaged

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions