Skip to content
This repository was archived by the owner on Sep 4, 2025. It is now read-only.

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Aug 4, 2025

The Grafana tool description was lacking the detail and clarity needed for optimal LLM tool selection, resulting in an estimated confidence score of ~0.3 (below the target of ≥0.5). This change significantly improves the description to provide comprehensive guidance on when and how to use the Grafana workspace operations tool.

Changes Made

Updated the tool description in GrafanaSetup.cs to address the following issues:

  • Enhanced specificity: Changed "resources" to "instances" for more precise terminology
  • Added comprehensive context: Included details about Azure Monitor, Application Insights, and other data sources integration
  • Clear usage guidance: Added explicit guidance on when to use the tool (listing workspaces, accessing configurations, managing permissions)
  • Defined boundaries: Clarified that this tool focuses on Azure Managed Grafana service management rather than dashboard content creation
  • Explicit exclusions: Added comprehensive guidance on when NOT to use the tool (Azure Monitor queries, Application Insights analytics, Log Analytics operations, creating dashboard content)

Before

Grafana workspace operations - Commands for managing and accessing Azure Managed Grafana resources and monitoring dashboards. Includes operations for listing Grafana workspaces and managing data visualization and monitoring capabilities.

After

Grafana workspace operations - Commands for managing and accessing Azure Managed Grafana instances for creating monitoring dashboards and data visualizations from Azure Monitor, Application Insights, and other data sources. Use this tool when you need to list Grafana workspaces, access workspace configurations, manage dashboard permissions, or retrieve information about available Grafana instances for monitoring and observability. This tool focuses on Azure Managed Grafana service management rather than dashboard content creation. Do not use this tool for Azure Monitor queries, Application Insights analytics, Log Analytics workspace operations, or creating dashboard content - this tool manages the Grafana service instances themselves.

The improved description provides the detail and clarity needed for optimal LLM tool selection while maintaining consistency with other tool description patterns in the codebase.

Fixes #827.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

@Copilot Copilot AI changed the title [WIP] [Tool Description] Improve Grafana tool description for better LLM selection [Tool Description] Improve Grafana tool description for better LLM selection Aug 4, 2025
@Copilot Copilot AI requested a review from charris-msft August 4, 2025 19:42
Copilot finished work on behalf of charris-msft August 4, 2025 19:42
@@ -0,0 +1,1888 @@
#!/usr/bin/env bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete the file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! I've removed the dotnet-install.sh file as requested. The file was inadvertently added during my exploration phase and should not have been included in the repository. Fixed in commit 57684b9.

Copilot finished work on behalf of xiangyan99 August 4, 2025 19:57
@Copilot Copilot AI requested a review from xiangyan99 August 4, 2025 19:57
@charris-msft charris-msft marked this pull request as ready for review August 5, 2025 18:37
@Copilot Copilot AI review requested due to automatic review settings August 5, 2025 18:37
@charris-msft charris-msft requested a review from a team as a code owner August 5, 2025 18:37
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves the Grafana tool description to enhance LLM tool selection accuracy by providing more comprehensive and specific guidance about when and how to use the Grafana workspace operations tool.

  • Updated tool description from generic "resources" terminology to more precise "instances"
  • Added detailed context about data source integrations and specific use cases
  • Included explicit boundaries and exclusions to prevent misuse of the tool

Copy link
Contributor

@charris-msft charris-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from a wording standpoint. Though I'm not clear on which features are supported.

@jongio
Copy link
Member

jongio commented Aug 25, 2025

Sorry for the inconvenience, but we have moved the Azure MCP Server source code to https://github.com/microsoft/mcp. This change allows us to build any Microsoft MCP server with the same engineering system and allows us to brand any MCP server with either Azure or Microsoft branding. This repo change means that you need to move this PR to that new repo. Please do so and then add a link here to that new PR. We'll close this PR now and we look forward to seeing this over on the new repo.

@jongio jongio closed this Aug 25, 2025
@github-project-automation github-project-automation bot moved this from Untriaged to Done in Azure MCP Server (OLD) Aug 25, 2025
@jongio jongio reopened this Aug 25, 2025
@jongio jongio requested a review from weng5e as a code owner August 25, 2025 18:59
@github-project-automation github-project-automation bot moved this from Done to Untriaged in Azure MCP Server (OLD) Aug 25, 2025
@jongio jongio closed this Aug 25, 2025
@github-project-automation github-project-automation bot moved this from Untriaged to Done in Azure MCP Server (OLD) Aug 25, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[Tool Description] Improve Grafana tool description for better LLM selection
4 participants