Skip to content

README.md

Latest commit

 

History

History
42 lines (32 loc) · 1.77 KB

README.md

File metadata and controls

42 lines (32 loc) · 1.77 KB
Raw

Codemod batch test

Runs the mcp-codemod v1 -> v2 migration against real, pinned repositories and audits the result with pyright, to find silent misses the unit tests and the in-repo example corpus cannot.

How it works

For each repository in repos.json:

  1. Clone the pinned commit (shallow).
  2. Run the codemod (sources and dependency files) over a copy.
  3. Type-check the pristine clone against an environment holding the latest v1 SDK, and the migrated copy against this workspace's v2 environment, with identical pyright settings.
  4. Diff the two error sets. Errors only on the migrated side are the migration surface; baseline noise (the repo's own issues, missing third-party stubs) appears on both sides and cancels out.
  5. Correlate each new error with the inserted # mcp-codemod: markers.

The codemod's contract is that the markers are the complete list of remaining manual work, so every new error should sit on or next to a marker. A new error with no nearby marker is a silent miss -- those are printed, written to work/results/<slug>.json, and make the run exit 1.

Usage

From the repository root (the v1 environment is created on first run):

uv run --frozen python scripts/codemod-batch-test/run.py            # all repos
uv run --frozen python scripts/codemod-batch-test/run.py --repo mcp-obsidian
uv run --frozen python scripts/codemod-batch-test/run.py --fresh    # re-clone

Adding a repository

Add an entry to repos.json with a pinned sha (never a branch), an include list when only part of the repository uses the SDK (empty means the whole tree), and a one-line note. Prefer repositories that depend on the mcp package directly; servers built on the external FastMCP library exercise that library's surface, not this SDK's.