E2E test for migration

[gaojude]

To prevent regression as we iterate on the prompts, we are adding some E2E tests to check that AI agents can produce a succesfully migrated app given the prompts and MCP tools.

Since the eval step is long and expensive (as it runs the actual agentic workflow from start to finish that costs real tokens), we do not run it in CI yet, and you need to supply your own Anthropic API key in order to run the pnpm test command.

Example test run output:

stdout | test/e2e/upgrade.test.ts > Next.js 14 → 16 Upgrade via MCP > should upgrade Next.js 14 project to Next.js 16 using Claude Agent SDK

✅ Completed after 72 messages

Verifying package.json has Next.js 16...
Running pnpm install...
Building upgraded project...
Test passed!

stdout | test/e2e/upgrade.test.ts > Next.js 14 → 16 Upgrade via MCP
Cleaning up temp directory...

 ✓ test/e2e/upgrade.test.ts (1 test) 273013ms
   ✓ Next.js 14 → 16 Upgrade via MCP > should upgrade Next.js 14 project to Next.js 16 using Claude Agent SDK  266328ms

 Test Files  1 passed (1)
      Tests  1 passed (1)
   Start at  01:46:24
   Duration  273.28s (transform 23ms, setup 0ms, collect 39ms, tests 273.01s, environment 0ms, prepare 38ms)

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	eslint-config-next@​14.2.33
	@​types/​react-dom@​18.3.7
	vitest@​3.2.4
	@​types/​react@​18.3.26
	next@​14.2.33
	@​anthropic-ai/​claude-agent-sdk@​0.1.15
	react@​18.3.1
	react-dom@​18.3.1
	dotenv@​17.2.3
	eslint@​8.57.1

View full report