Concurrent MCP discovery + robust schema normalization for Harmony tool injection

[mc936h]

Concurrent MCP discovery + robust schema normalization for Harmony tool injection

Summary

• Fetch MCP servers concurrently (no more asyncio.run per URL).
• Normalize MCP JSON Schemas to Harmony’s variant:
  • Handles anyOf / oneOf / allOf / nullable
  • Deep-copies schemas (no in-place mutation)
  • Preserves enum, description, format
• Resilient to partial failures — one bad server no longer breaks the build.
• Deterministic UTC conversation_start_date.
• Clearer variable naming, logging, and light type hints.
• Centralized tool URL config for easier updates.

Why

• Latency: Parallel discovery reduces startup time from the sum of all server latencies to the max of them.
• Correctness: Tools with oneOf / nullable previously broke or were dropped; now they’re usable.
• Reliability: Script continues even if one local tool server is down.
• Maintainability: Easier to reason about, test, and extend.

Key changes

• Added helpers:
  • _normalize_schema — deep-copy schema normalizer with extended support for common JSON Schema patterns.
  • _filter_tools — removes tools excluded from prompts with safe annotation checks.
  • gather_servers — fetches all MCP servers concurrently.
  • build_harmony_system_content — cleanly constructs SystemContent with all discovered tools.
• Logging on exclusions & failures.
• Uses UTC date via datetime.now(timezone.utc).
• Avoids variable shadowing (no reusing system_message).
• More explicit output variable: rendered_system_text.

Before / After

Before

• Sequential SSE calls; one failure aborted the run.
• Only anyOf support; oneOf / nullable unsupported.
• Mutated schema in-place.
• Local time in system message; ambiguous output variable.

After

• Concurrency via asyncio.gather.
• Broader schema support; deep-copy; preserves key metadata.
• UTC date; explicit rendered_system_text output.

Tests

Unit (new)

• _normalize_schema with:
  • anyOf: [{"type":"string"},{"type":"null"}] → type: ["string"]
  • oneOf → flattened type list
  • allOf simple merges
  • nullable: true → type includes "null"
  • Recursion into properties / items
• _filter_tools respects annotations.include_in_prompt

Manual (local)

1. Start browser and python MCP servers at :8001 / :8000.
2. Run script twice: once with both servers running; once with one stopped — verify output is still valid.
3. Introduce a tool with oneOf and enum; ensure it appears correctly.

Rollout / Risk

• Low risk: only changes how the system message is built; runtime consumers unchanged.
• Possible limitation: highly nested/complex allOf merges are naive — logged as warnings.

Follow-ups

• Add env/CLI config for server URLs (MCP_SSE_URLS) and REASONING_EFFORT.
• Validate tool schemas against a “Harmony schema” using jsonschema or pydantic.
• Add metrics: tools per namespace, excluded counts, build latency.
• Golden-file test for rendered system text.

Reviewer checklist

• Concurrency logic is correct — only one asyncio.run call.
• Schema normalization does not drop meaningful fields.
• Partial failures still yield usable SystemContent.
• Logging is clear and helpful.

[mc936h]

@romainhuet can you please review this PR and provide some feedback?

Thank you!