test: upgrade flow testing

[itsjustriley]

WHY are these changes introduced?

Fixes https://github.com/Shopify/developer-tools-team/issues/1045

The existing upgrade-flow.test.ts was unreliable. Tests would intermittently fail in CI due to dev server flakiness (port conflicts, spawn/kill race conditions, HTTP timeouts), making it difficult to validate changelog.json entries and creating toil for releases.

WHAT is this pull request doing?

Replaces upgrade-flow.test.ts (1313 lines) with upgrade-e2e.test.ts (719 lines) that:

Fixes flakiness by:

• ✅ Removing dev server tests → validates with npm run build instead (deterministic)
• ✅ Using inTemporaryDirectory → auto-cleanup (no temp dir leaks)
• ✅ Using vi.stubEnv() → proper env var cleanup between tests
• ✅ Failing when packages aren't published → catches changelog errors (was silent skip)

Tests upgrade paths by:

• Scaffolding real historical projects from git (via git archive)
• Running actual h2 upgrade command
• Validating dependencies/devDependencies update correctly
• Running npm install to catch peer dependency conflicts
• Running npm run build for clean upgrades
• Validating upgrade guide generation for manual step releases (target or intermediate)
• Skipping paths with breaking changes (target or intermediate)
• Stopping early in LAST_N (breaking changes are cumulative)

Test matrix support:

• UPGRADE_TEST_FROM=<version> - Test from specific version
• UPGRADE_TEST_TO=<version> - Test to specific version
• UPGRADE_TEST_LAST_N=<number> - Test last N versions

HOW to test your changes?

All Test Scenarios & Expected Behavior

Scenario	Changelog Characteristics	What Gets Validated	Expected Output
Clean Upgrade	No breaking: true in target or intermediate versions No manual steps	✅ Deps update ✅ npm install ✅ npm run build	Testing: 2025.10.0 → 2025.10.1 ✓ 2025.10.0 → 2025.10.1
Manual Steps	Target or intermediate versions have steps array No breaking changes	✅ Deps update ✅ npm install ✅ Guide generates ⊘ Build skipped	Testing: 2025.4.2 → 2025.5.1 ✓ 2025.4.2 → 2025.5.1 [manual steps required, build skipped]
Breaking Changes	Target or intermediate versions have breaking: true	⊘ Path skipped (no automatic upgrade exists)	⊘ Skipping 2025.10.1: target has breaking changes OR ⊘ Skipping X → Y: intermediate breaking changes
Missing Packages	Packages not yet on npm	❌ Test fails fast (before upgrade runs)	Error: Required packages not yet published to npm: @shopify/hydrogen@X.X.X.
Intermediate Manual Steps	Neither from nor to has steps, but an intermediate version does	✅ Deps update ✅ npm install ✅ Guide generates ⊘ Build skipped	Testing: 2025.1.2 → 2025.4.1 ✓ 2025.1.2 → 2025.4.1 [manual steps required, build skipped]
Incomplete Changelog	Missing deps/removeDeps	❌ npm install fails	Error: npm install failed for X → Y. This indicates the upgrade path doesn't work automatically
Multiple Versions (LAST_N)	Testing N paths to same target	Tests multiple FROM versions (stops at first breaking change)	Testing last 3: 2025.7.0 → 2025.10.1 ✓ 2025.10.0 → 2025.10.1 ⊘ Skipping 2025.7.1 → 2025.10.1: intermediate breaking changes ⊘ Stopped early due to breaking changes (1 remaining path would also skip)
Hash Unreachable	changelog.json hash exists but not in local git history (squashed/shallow clone)	⚠️ Warning logged, falls back to git search	⚠️ Hash abc123 for 2025.10.0 not found in local git history. Falling back to git search (slower).

Tophatting Commands

cd packages/cli

# 1. Default (previous → latest)
# Latest (2026.1.0) has breaking changes
# Expect: ⊘ Skipping 2026.1.0: target has breaking changes
npm test upgrade-e2e.test.ts

# 2. Single version jump (adjacent non-breaking versions)
# Expect: ✓ 2025.10.0 → 2025.10.1
UPGRADE_TEST_FROM=2025.10.0 UPGRADE_TEST_TO=2025.10.1 npm test upgrade-e2e.test.ts

# 3. Multi-version jump (intermediate breaking changes)
# Expect: ⊘ Skipping 2025.1.0 → 2025.10.1: intermediate breaking changes
UPGRADE_TEST_FROM=2025.1.0 UPGRADE_TEST_TO=2025.10.1 npm test upgrade-e2e.test.ts

# 4. Last N versions (matrix testing, stops at first breaking intermediate)
# Expect: ✓ 2025.10.0 → 2025.10.1, then stops at 2025.7.1 (2025.10.0 is breaking intermediate)
UPGRADE_TEST_TO=2025.10.1 UPGRADE_TEST_LAST_N=3 npm test upgrade-e2e.test.ts

# 5. Catches incomplete changelog entry (demonstrates test finding a real bug)
# 2025.1.3 introduced vite 5 → 6 but is not marked breaking: true, so the
# scaffolded 2025.1.2 project fails npm install due to a peer dep conflict.
# Expect: ❌ npm install failed (fix: add breaking: true to 2025.1.3 in changelog.json)
#
# To verify the skip logic works: temporarily add `"breaking": true` to any fix/feature
# in the 2025.1.3 entry in docs/changelog.json, then re-run. The test reads the local
# changelog (FORCE_CHANGELOG_SOURCE=local) so no publish needed.
# Expect: ⊘ Skipping 2025.1.2 → 2025.4.1: intermediate breaking changes
UPGRADE_TEST_FROM=2025.1.2 UPGRADE_TEST_TO=2025.4.1 npm test upgrade-e2e.test.ts

Verify CI

1. Modify docs/changelog.json (add whitespace)
2. Push to branch
3. Check test-upgrade-flow.yml workflow passes

Post-merge steps

None required.

Checklist

• I've read the Contributing Guidelines
• I've considered possible cross-platform impacts (Mac, Linux, Windows) - This is designed to function in CI/on Macs as it is written for Shopify maintainers
• I've added a changeset - Not needed (test infrastructure only)
• I've added tests to cover my changes
• I've added or updated the documentation

[shopify]

Oxygen deployed a preview of your upgrade-flow-tests-new branch. Details:

Storefront	Status	Preview link	Deployment details	Last update (UTC)
Skeleton (skeleton.hydrogen.shop)	✅ Successful (Logs)	Preview deployment	Inspect deployment	February 26, 202611:35 AM

Learn more about Hydrogen's GitHub integration.