-
Notifications
You must be signed in to change notification settings - Fork 338
Improve Kusto tool description for better LLM selection #835
Conversation
Co-authored-by: xiangyan99 <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR improves the Kusto tool description to provide better guidance for LLM tool selection by expanding from a basic 200-character functional description to a comprehensive 1000+ character guide. The enhanced description addresses key deficiencies by adding clear usage guidance, negative guidance for when not to use the tool, and enhanced technical context.
- Added specific use cases and scenarios for when to use the Kusto tool
- Added explicit guidance on when NOT to use this tool to prevent confusion with other Azure services
- Enhanced technical context about Azure Data Explorer, KQL, and permission requirements
@@ -20,7 +20,7 @@ public void ConfigureServices(IServiceCollection services) | |||
public void RegisterCommands(CommandGroup rootGroup, ILoggerFactory loggerFactory) | |||
{ | |||
// Create Kusto command group | |||
var kusto = new CommandGroup("kusto", "Kusto operations - Commands for managing and querying Azure Data Explorer (Kusto) resources. Includes operations for listing clusters and databases, executing KQL queries, retrieving table schemas, and working with Kusto data analytics workloads."); | |||
var kusto = new CommandGroup("kusto", "Kusto operations - Commands for managing and querying Azure Data Explorer (Kusto) clusters for big data analytics, log analysis, and telemetry processing using KQL (Kusto Query Language). Use this tool when you need to list Data Explorer clusters and databases, execute KQL queries against large datasets, retrieve table schemas, analyze time-series data, or work with high-volume analytics workloads from IoT, applications, or infrastructure logs. This tool is ideal for complex analytics queries and data exploration scenarios. Do not use this tool for transactional database operations, simple key-value lookups, Azure SQL Database queries, or Azure Monitor Log Analytics workspace operations - use the appropriate database or monitor tools instead. This tool is a hierarchical MCP command router where sub-commands are routed to MCP servers that require specific fields inside the \"parameters\" object. To invoke a command, set \"command\" and wrap its arguments in \"parameters\". Set \"learn=true\" to discover available sub-commands for different Kusto cluster and query operations. Note that this tool requires appropriate Azure Data Explorer permissions and will only access clusters and data accessible to the authenticated user."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This description string is extremely long (over 1000 characters) and difficult to read in the code. Consider extracting it to a constant or resource file, or breaking it into multiple concatenated strings for better maintainability.
var kusto = new CommandGroup("kusto", "Kusto operations - Commands for managing and querying Azure Data Explorer (Kusto) clusters for big data analytics, log analysis, and telemetry processing using KQL (Kusto Query Language). Use this tool when you need to list Data Explorer clusters and databases, execute KQL queries against large datasets, retrieve table schemas, analyze time-series data, or work with high-volume analytics workloads from IoT, applications, or infrastructure logs. This tool is ideal for complex analytics queries and data exploration scenarios. Do not use this tool for transactional database operations, simple key-value lookups, Azure SQL Database queries, or Azure Monitor Log Analytics workspace operations - use the appropriate database or monitor tools instead. This tool is a hierarchical MCP command router where sub-commands are routed to MCP servers that require specific fields inside the \"parameters\" object. To invoke a command, set \"command\" and wrap its arguments in \"parameters\". Set \"learn=true\" to discover available sub-commands for different Kusto cluster and query operations. Note that this tool requires appropriate Azure Data Explorer permissions and will only access clusters and data accessible to the authenticated user."); | |
var kusto = new CommandGroup("kusto", KustoCommandGroupDescription); |
Copilot uses AI. Check for mistakes.
Sorry for the inconvenience, but we have moved the Azure MCP Server source code to https://github.com/microsoft/mcp. This change allows us to build any Microsoft MCP server with the same engineering system and allows us to brand any MCP server with either Azure or Microsoft branding. This repo change means that you need to move this PR to that new repo. Please do so and then add a link here to that new PR. We'll close this PR now and we look forward to seeing this over on the new repo. |
Enhanced the Kusto tool description to provide better guidance for LLM tool selection by addressing several key deficiencies in the original description.
Changes Made
Updated the tool description in
KustoSetup.cs
from a basic functional description to a comprehensive guide that includes:Added Usage Guidance:
Added Negative Guidance:
Enhanced Technical Context:
Before vs After
Before (200 characters):
After (1000+ characters):
Comprehensive description covering use cases, when to use vs not use, technical details, and permission requirements.
This improvement should increase LLM tool selection confidence score from ~0.5 to >0.6 by providing clearer context about when this tool is appropriate versus other Azure database and analytics tools.
Fixes #830.
💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.