-
Notifications
You must be signed in to change notification settings - Fork 338
[Tool Description] Improve Foundry tool description for better LLM selection #836
Conversation
Co-authored-by: xiangyan99 <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR enhances the Azure AI Foundry tool description to improve LLM tool selection accuracy by providing comprehensive context, specific service details, and clear usage guidance. The description was expanded from a brief 80-character summary to a detailed 1,091-character explanation to achieve better LLM confidence scores.
Key Changes:
- Enhanced tool description with Azure AI Foundry branding and specific service identification
- Added detailed usage guidance including when to use and when not to use the tool
- Improved technical documentation for MCP command router functionality
@@ -19,7 +19,7 @@ public void ConfigureServices(IServiceCollection services) | |||
|
|||
public void RegisterCommands(CommandGroup rootGroup, ILoggerFactory loggerFactory) | |||
{ | |||
var foundry = new CommandGroup("foundry", "Foundry service operations - Commands for listing and managing services and resources in AI Foundry."); | |||
var foundry = new CommandGroup("foundry", "Foundry service operations - Commands for listing and managing Azure AI Foundry services and resources including AI models, endpoints, deployments, and workspace configurations. Use this tool when you need to work with Azure AI Foundry projects, list available AI models, manage model deployments, configure AI endpoints, or query AI workspace resources and settings. This tool is designed for AI and machine learning workflow management within Azure AI Foundry environments. Do not use this tool for general Azure AI services outside of Foundry, Azure Machine Learning studio operations, or Azure OpenAI direct management - this tool specifically targets Azure AI Foundry service operations. This tool is a hierarchical MCP command router where sub-commands are routed to MCP servers that require specific fields inside the \"parameters\" object. To invoke a command, set \"command\" and wrap its arguments in \"parameters\". Set \"learn=true\" to discover available sub-commands for different AI Foundry operations and resource types."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This extremely long string literal (1,091 characters) should be extracted to a constant or broken into multiple concatenated strings for better readability and maintainability. Consider using a multi-line string with proper formatting.
var foundry = new CommandGroup("foundry", "Foundry service operations - Commands for listing and managing Azure AI Foundry services and resources including AI models, endpoints, deployments, and workspace configurations. Use this tool when you need to work with Azure AI Foundry projects, list available AI models, manage model deployments, configure AI endpoints, or query AI workspace resources and settings. This tool is designed for AI and machine learning workflow management within Azure AI Foundry environments. Do not use this tool for general Azure AI services outside of Foundry, Azure Machine Learning studio operations, or Azure OpenAI direct management - this tool specifically targets Azure AI Foundry service operations. This tool is a hierarchical MCP command router where sub-commands are routed to MCP servers that require specific fields inside the \"parameters\" object. To invoke a command, set \"command\" and wrap its arguments in \"parameters\". Set \"learn=true\" to discover available sub-commands for different AI Foundry operations and resource types."); | |
var foundry = new CommandGroup("foundry", FoundryServiceOperationsDescription); |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot The description character count should be at most 1024 characters long. Please make it a bit shorter while keeping the meaning and main concepts as intact as possible.
Sorry for the inconvenience, but we have moved the Azure MCP Server source code to https://github.com/microsoft/mcp. This change allows us to build any Microsoft MCP server with the same engineering system and allows us to brand any MCP server with either Azure or Microsoft branding. This repo change means that you need to move this PR to that new repo. Please do so and then add a link here to that new PR. We'll close this PR now and we look forward to seeing this over on the new repo. |
The Foundry tool description was lacking the detail and clarity needed for optimal LLM tool selection, resulting in an estimated confidence score of ~0.2 (well below target of ≥0.5).
Issues Addressed
The previous description was too brief and vague:
Problems:
Solution
Enhanced the description to provide comprehensive context and guidance:
Key Improvements
Impact
Fixes #826.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.