Skip to content
This repository was archived by the owner on Sep 4, 2025. It is now read-only.

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Aug 4, 2025

Enhanced the Kusto tool description to provide better guidance for LLM tool selection by addressing several key deficiencies in the original description.

Changes Made

Updated the tool description in KustoSetup.cs from a basic functional description to a comprehensive guide that includes:

Added Usage Guidance:

  • Clear use cases: big data analytics, log analysis, and telemetry processing using KQL
  • Specific scenarios: analyzing time-series data, high-volume analytics workloads from IoT, applications, or infrastructure logs
  • Ideal for complex analytics queries and data exploration scenarios

Added Negative Guidance:

  • Explicitly states when NOT to use this tool
  • Clarifies distinction from Azure SQL Database operations
  • Distinguishes from Azure Monitor Log Analytics workspace operations
  • Excludes transactional database operations and simple key-value lookups

Enhanced Technical Context:

  • Expanded explanation of Azure Data Explorer's role in big data analytics
  • Added details about KQL (Kusto Query Language)
  • Clarified hierarchical MCP command router functionality
  • Added permission requirements and accessibility notes

Before vs After

Before (200 characters):

Kusto operations - Commands for managing and querying Azure Data Explorer (Kusto) resources. Includes operations for listing clusters and databases, executing KQL queries, retrieving table schemas, and working with Kusto data analytics workloads.

After (1000+ characters):
Comprehensive description covering use cases, when to use vs not use, technical details, and permission requirements.

This improvement should increase LLM tool selection confidence score from ~0.5 to >0.6 by providing clearer context about when this tool is appropriate versus other Azure database and analytics tools.

Fixes #830.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

@Copilot Copilot AI changed the title [WIP] [Tool Description] Improve Kusto tool description for better LLM selection Improve Kusto tool description for better LLM selection Aug 4, 2025
@Copilot Copilot AI requested a review from xiangyan99 August 4, 2025 20:16
Copilot finished work on behalf of xiangyan99 August 4, 2025 20:16
@xiangyan99 xiangyan99 marked this pull request as ready for review August 4, 2025 20:38
@Copilot Copilot AI review requested due to automatic review settings August 4, 2025 20:38
@xiangyan99 xiangyan99 requested a review from a team as a code owner August 4, 2025 20:38
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves the Kusto tool description to provide better guidance for LLM tool selection by expanding from a basic 200-character functional description to a comprehensive 1000+ character guide. The enhanced description addresses key deficiencies by adding clear usage guidance, negative guidance for when not to use the tool, and enhanced technical context.

  • Added specific use cases and scenarios for when to use the Kusto tool
  • Added explicit guidance on when NOT to use this tool to prevent confusion with other Azure services
  • Enhanced technical context about Azure Data Explorer, KQL, and permission requirements

@@ -20,7 +20,7 @@ public void ConfigureServices(IServiceCollection services)
public void RegisterCommands(CommandGroup rootGroup, ILoggerFactory loggerFactory)
{
// Create Kusto command group
var kusto = new CommandGroup("kusto", "Kusto operations - Commands for managing and querying Azure Data Explorer (Kusto) resources. Includes operations for listing clusters and databases, executing KQL queries, retrieving table schemas, and working with Kusto data analytics workloads.");
var kusto = new CommandGroup("kusto", "Kusto operations - Commands for managing and querying Azure Data Explorer (Kusto) clusters for big data analytics, log analysis, and telemetry processing using KQL (Kusto Query Language). Use this tool when you need to list Data Explorer clusters and databases, execute KQL queries against large datasets, retrieve table schemas, analyze time-series data, or work with high-volume analytics workloads from IoT, applications, or infrastructure logs. This tool is ideal for complex analytics queries and data exploration scenarios. Do not use this tool for transactional database operations, simple key-value lookups, Azure SQL Database queries, or Azure Monitor Log Analytics workspace operations - use the appropriate database or monitor tools instead. This tool is a hierarchical MCP command router where sub-commands are routed to MCP servers that require specific fields inside the \"parameters\" object. To invoke a command, set \"command\" and wrap its arguments in \"parameters\". Set \"learn=true\" to discover available sub-commands for different Kusto cluster and query operations. Note that this tool requires appropriate Azure Data Explorer permissions and will only access clusters and data accessible to the authenticated user.");
Copy link
Preview

Copilot AI Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description string is extremely long (over 1000 characters) and difficult to read in the code. Consider extracting it to a constant or resource file, or breaking it into multiple concatenated strings for better maintainability.

Suggested change
var kusto = new CommandGroup("kusto", "Kusto operations - Commands for managing and querying Azure Data Explorer (Kusto) clusters for big data analytics, log analysis, and telemetry processing using KQL (Kusto Query Language). Use this tool when you need to list Data Explorer clusters and databases, execute KQL queries against large datasets, retrieve table schemas, analyze time-series data, or work with high-volume analytics workloads from IoT, applications, or infrastructure logs. This tool is ideal for complex analytics queries and data exploration scenarios. Do not use this tool for transactional database operations, simple key-value lookups, Azure SQL Database queries, or Azure Monitor Log Analytics workspace operations - use the appropriate database or monitor tools instead. This tool is a hierarchical MCP command router where sub-commands are routed to MCP servers that require specific fields inside the \"parameters\" object. To invoke a command, set \"command\" and wrap its arguments in \"parameters\". Set \"learn=true\" to discover available sub-commands for different Kusto cluster and query operations. Note that this tool requires appropriate Azure Data Explorer permissions and will only access clusters and data accessible to the authenticated user.");
var kusto = new CommandGroup("kusto", KustoCommandGroupDescription);

Copilot uses AI. Check for mistakes.

@jongio
Copy link
Member

jongio commented Aug 25, 2025

Sorry for the inconvenience, but we have moved the Azure MCP Server source code to https://github.com/microsoft/mcp. This change allows us to build any Microsoft MCP server with the same engineering system and allows us to brand any MCP server with either Azure or Microsoft branding. This repo change means that you need to move this PR to that new repo. Please do so and then add a link here to that new PR. We'll close this PR now and we look forward to seeing this over on the new repo.

@jongio jongio closed this Aug 25, 2025
@github-project-automation github-project-automation bot moved this from Untriaged to Done in Azure MCP Server (OLD) Aug 25, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[Tool Description] Improve Kusto tool description for better LLM selection
3 participants