[WIP- Do Not Review] Foundry mcp evaluation

[needuv]

What does this PR do?

[Provide a clear, concise description of the changes]

[Any additional context, screenshots, or information that helps reviewers]

GitHub issue number?

[Link to the GitHub issue this PR addresses]

Pre-merge Checklist

• Required for All PRs
  • Read contribution guidelines
  • PR title clearly describes the change
  • Commit history is clean with descriptive messages (cleanup guide)
  • Added comprehensive tests for new/modified functionality
  • Updated CHANGELOG.md for product changes (features, bug fixes, UI/UX, updated dependencies)
  • Spelling check passes: .\eng\common\spelling\Invoke-Cspell.ps1
• For MCP tool changes, updated:
  • Updated README.md documentation
  • Updated command list in /docs/azmcp-commands.md
  • Updated test prompts in /e2eTests/e2eTestPrompts.md
• 👉 For Community (non-Azure team member) PRs:
  • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
  • Manual tests run: added comment /azp run azure - mcp to run Live Test Pipeline

[Copilot]

Pull Request Overview

This PR adds Azure AI Foundry evaluation capabilities including agent management and evaluation tools. It introduces comprehensive functionality for connecting to agents, running queries, and performing evaluations using various built-in evaluators.

Key changes include:

• Added new evaluation commands for agent listing, connection, querying, and evaluation
• Integrated Microsoft.Extensions.AI evaluation libraries with safety and quality evaluators
• Added support for Azure AI Agents with persistent agent functionality

Reviewed Changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/AzureMcp.csproj	Added Azure AI and evaluation package references
src/Areas/Foundry/Services/IFoundryService.cs	Extended interface with agent operations
src/Areas/Foundry/Services/FoundryService.cs	Implemented agent connection, querying, and evaluation logic
src/Areas/Foundry/Commands/Evaluation/*.cs	Added command implementations for agent operations
src/Areas/Foundry/Options/Evaluation/*.cs	Added option classes for evaluation commands
src/Areas/Foundry/FoundrySetup.cs	Registered new evaluation commands
tests/Areas/Foundry/UnitTests/*.cs	Added unit tests for new commands
tests/Areas/Foundry/LiveTests/FoundryCommandTests.cs	Added placeholder live tests
Directory.Packages.props	Updated package versions and added new dependencies
AzureMcp.sln	Solution file with extraneous characters

Comments suppressed due to low confidence (3)

tests/Areas/Foundry/LiveTests/FoundryCommandTests.cs:78

• The test methods contain placeholder comments but don't actually test the new agent functionality. These tests should be implemented or marked as skipped.

        // to be filled in

Directory.Packages.props:41

• This appears to be a preview package version that may not exist or be stable. Verify this version exists and consider using a stable release if available.

    <PackageVersion Include="Microsoft.Extensions.AI.Evaluation.Safety" Version="9.7.0-preview.1.25356.2" />

Directory.Packages.props:42

• This appears to be a preview package version that may not exist or be stable. Verify this version exists and consider using a stable release if available.

    <PackageVersion Include="Microsoft.Extensions.AI.OpenAI" Version="9.7.0-preview.1.25356.2" />

[kvenkatrajan]

I was wondering if it would be useful to have tools/commands to access some of the manage options (screenshot below)- view quota, connect services, auth/permissions for new apps connecting to the agents etc:

CC: @sandeep-sen , @charris-msft

[jongio]

Let's get this opened against new repo next week.