feat: Migrate eval from MCPStdio to mcpd (fixes #279)

[anivar]

Summary

Fixes #279 - Migrate evaluation system from MCPStdio to mcpd

Changes

• Created mcpd_tools.py module to bridge any-agent with mcpd's REST API
• Updated generate_evaluation_case.py to use mcpd instead of MCPStdio
• Updated example in instructions.py to show mcpd usage

Implementation Details

Since mcpd doesn't support WebSocket/streamable-http protocol, this PR implements a REST API bridge:

1. Static Tools (Default): Hardcoded filesystem tools for immediate functionality
2. Dynamic Discovery: Optional --use_dynamic_discovery flag fetches schemas from mcpd
3. Full Replacement: create_all_server_tools() can replace mcpd Python client

Usage Comparison: MCPStdio vs mcpd

Aspect	MCPStdio (Before)	mcpd (After)
Import	from any_agent.config import MCPStdio	from eval.mcpd_tools import create_filesystem_tools
Protocol	Process-based (stdio)	REST API
Transport	Subprocess communication	HTTP POST to /api/v1/servers/{server}/tools/{tool}
Tool Discovery	Static list in config	Dynamic via /api/v1/servers/{server}/tools endpoint
Dependencies	Docker/subprocess	mcpd daemon running
Error Handling	Process crashes	HTTP errors with graceful fallback

Configuration Example

Before (MCPStdio):

MCPStdio(
    command="docker",
    args=["run", "-i", "--rm", "mcp/filesystem"],
    env={"PATH": "/workspace"},
    tools=["read_file", "list_directory"]
)

After (mcpd):

# Static (default)
filesystem_tools = create_filesystem_tools(mcpd_url)

# Dynamic discovery
await create_server_tools(
    "filesystem",
    ["read_file", "list_directory"], 
    mcpd_url
)

Usage

# Default (static tools)
python eval/generate_evaluation_case.py

# With dynamic discovery
python eval/generate_evaluation_case.py --use_dynamic_discovery=true

Future Compatibility

Pre-configured for AGNTCY protocol support (mozilla-ai/mcpd#154). When merged, just uncomment the DID header injection code in mcpd_call_tool().

[dpoulopoulos]

Hey @anivar, thanks for this contribution. Since the evaluation step is dependent only on one MCP server (the filesystem one) we chose to not over-complicate things by introducing mcpd here. This would mean that any user would have to run an mcpd daemon just to serve one MCP server over HTTP. So, for this case, we're keeping MCPStdio.

[anivar]

@dpoulopoulos Thanks for the thoughtful feedback! You make a valid point about avoiding unnecessary complexity.