feat: add a local dev script for testing MCP tools without manual curl handshake

[shahfazal]

Testing tools locally currently requires manually executing the full MCP protocol handshake via curl — three sequential steps, with session ID extraction between calls:

Step 1 — initialize (capture session ID from response header)

SESSION_ID=$(curl -s -D - -X POST http://localhost:8000/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "Host: localhost" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"0.1"}}}' \
  | grep -i "mcp-session-id" | awk '{print $2}' | tr -d '\r')

Step 2 — confirm initialized

curl -s -X POST http://localhost:8000/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "Host: localhost" \
  -H "mcp-session-id: $SESSION_ID" \
  -d '{"jsonrpc":"2.0","method":"notifications/initialized","params":{}}'

Step 3 — call a tool

curl -s -X POST http://localhost:8000/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "Host: localhost" \
  -H "mcp-session-id: $SESSION_ID" \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"search_datasets","arguments":{"query":"IRVE"}}}'

This is a significant friction point for contributors who want to quickly verify tool behaviour after making changes. It's also error-prone: a missing Host header, a dropped session ID, or an out-of-order step produces cryptic errors with no obvious explanation.

Proposal

Add a scripts/call_tool.py dev helper that wraps the full handshake using the MCP Python client SDK — which is already a dependency of this project (mcp>=1.25.0), so no new packages are needed.

The mcp package ships mcp.client.streamable_http.streamable_http_client and mcp.client.session.ClientSession, which handle protocol negotiation, session management, and SSE parsing automatically.

A minimal script could look like:

# scripts/call_tool.py
# Usage: python scripts/call_tool.py search_datasets '{"query": "IRVE", "sort": "-created"}'
import asyncio, json, sys
from mcp.client.streamable_http import streamable_http_client
from mcp.client.session import ClientSession

async def main(tool_name: str, args: dict):
    async with streamable_http_client("http://localhost:8000/mcp") as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()
            result = await session.call_tool(tool_name, args)
            print(result.content[0].text)

tool = sys.argv[1]
arguments = json.loads(sys.argv[2]) if len(sys.argv) > 2 else {}
asyncio.run(main(tool, arguments))

Contributors could then run:

python scripts/call_tool.py search_datasets '{"query": "IRVE", "sort": "-created"}'
python scripts/call_tool.py get_resource_info '{"resource_id": "..."}'

Benefits

Zero new dependencies — uses the mcp package already installed
No session management or header wrangling for contributors
Quicker feedback loop when developing or debugging a tool
Could double as a smoke-test reference for the README

Happy to implement this — I already have a PR open for this project (#20) and would be glad to contribute the test script as a follow-up. Let me know if the approach looks good or if there's a preferred location/naming convention.

[bolinocroustibat]

This is a great idea to improve DX. That being said, following the recent incident, our most urgent need right now is to set up a very simple yet robust monitoring system to detect downtime.

Do you think a similar logic could be integrated into our /health endpoint? Alternatively, could we use this script as a standalone probe to continuously verify that the full MCP handshake and tool execution are functioning correctly?

[shahfazal]

(apologies for the delay in responding, I'm currently traveling)

Regarding monitoring, it's a valid point. A simple /health check can be 'green' while the MCP logic is actually 'red' — like what happened in the recent incident.

Looking at the current code:

• /health returns OK + timestamp + version (lightweight, as it should be)
• Matomo tracking gives you usage analytics but isn't designed for operational monitoring
• Looks like there's no deep protocol health checks yet

I can adapt the script into a standalone health probe that fills this gap, namely a

Standalone probe approach:

• Exercises full MCP handshake + lightweight tool call (list_datasets with limit=1)
• Strict timeouts (5-10s max)
• Standard exit codes: 0 for OK, 2 for Critical
• Structured logging (JSON output for easy parsing)
• Can be run via cron or monitoring agent (every 1-5 minutes)

Health probe mode

python scripts/call_tool.py --health-check list_datasets '{"limit": 1}'

Dev iteration mode (original goal)

python scripts/call_tool.py search_datasets '{"query": "IRVE"}'

Architecture decision: I think we can keep /health lightweight as-is (fast, no side effects) and running the deep probe externally. This separates concerns and lets you control check frequency independently without adding latency to your health endpoint.

Happy to draft this — does this approach align with how you'd want to monitor it?

[bolinocroustibat]

@shahfazal Ah, I understand better now! When you mentioned the --health-check flag and returning exit codes, you were thinking of using call_tool.py as the HEALTHCHECK command in our Docker setup? That makes total sense for letting Docker continuously know if the container is running properly.

At the same time, I really want to prioritize making our HTTP /health endpoint do "deep" monitoring. External monitoring tools and dashboards will constantly ping this HTTP route to check uptime, and they can't rely on Docker.

So, I see these as two different features that share the same goal. We could create a single internal test (e.g., trying to call list_datasets(limit=1) as you rightly suggest), and then wire it up to both:

Your proposal: call_tool.py --health-check runs the test and returns exit 0 or exit 1 for Docker.

My proposal: The /health HTTP route runs the exact same test and returns 200 OK or 503 Service Unavailable for external monitors.

We can start iteratively with either one, but what do you think of this dual approach for the end goal?

[shahfazal]

@bolinocroustibat Once again, apologies for the delay in responding—travel madness done, I'm finally back stateside and ready to move this forward.

Agreed on the dual approach — sharing the same probe logic is the right call. One internal function (e.g., _run_deep_check()) that both consumers avoid code repetition, and ensures Docker and external monitors are testing the exact same thing.

Proposed split:

scripts/call_tool.py— dev iteration + --health-check mode for Docker
helpers/health_probe.py (or inline in main.py) — same check, called from the /health route, returns 503 on failure

I could start with the script (simpler, no server changes needed) and then wire up the /health endpoint in the same PR. Does that scope work for you?