Skip to content

feat(vitest): add --stale CLI option for iterative test running#9916

Open
holyhli wants to merge 8 commits intovitest-dev:mainfrom
holyhli:feat/stale-option
Open

feat(vitest): add --stale CLI option for iterative test running#9916
holyhli wants to merge 8 commits intovitest-dev:mainfrom
holyhli:feat/stale-option

Conversation

@holyhli
Copy link
Copy Markdown

@holyhli holyhli commented Mar 19, 2026

Description

Adds a --stale CLI option inspired by Elixir Mix's --stale option that runs only test files whose transitive module dependencies have changed since the last --stale run.

Primary Use Case - Agentic Coding Systems: Modern agentic coding tools (Claude Code, GitHub Copilot Workspace, OpenCode, Cursor with Composer, etc.) build harness systems that rely heavily on unit tests for validation. These agents run tests continuously during their development loops, and running full test suites dramatically slows their iteration time. --stale provides the fast feedback cycle agents need by running only affected tests.

Secondary Use Case - Human Development: Developers making small iterative changes also benefit from faster test cycles without requiring git.

How it works:

  • Uses filesystem modification times (mtimes) stored in a manifest
  • Tracks transitive dependencies recursively via Vite's module graph
  • First run: executes all tests, generates manifest
  • Subsequent runs: only runs tests with changed dependencies

Resolves #9917

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. ✅ See feat: add --stale CLI option for agentic coding systems #9917 for feature discussion
  • Ideally, include a test that fails without this PR but passes with it. ✅ Added 6 comprehensive tests in test/cli/test/stale.test.ts
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example. ✅ No changes to pnpm-lock.yaml
  • Please check Allow edits by maintainers to make review process faster. ← Please enable this on GitHub

Tests

  • Run the tests with pnpm test:ci. ✅ All tests pass:
    • CI=true pnpm test stale.test.ts — 6/6 tests pass
    • CI=true pnpm test git-changed.test.ts — 5/5 tests pass (no regression)
    • pnpm build — All 17 packages build successfully
    • pnpm typecheck — Zero type errors

Documentation

  • If you introduce new functionality, document it. ✅ Documentation added:
    • Manual prose section in docs/guide/cli.md
    • Auto-generated CLI table in docs/guide/cli-generated.md
    • JSDoc comments in code

Changesets

  • Changes in changelog are generated from PR name. ✅ PR title: feat(vitest): add --stale CLI option for iterative test running
    • Prefix: feat: ✓
    • Clear description ✓
    • Breaking changes: None

Implementation Highlights

Agentic System Benefits:

  • Fast feedback loops: Only affected tests run → agents iterate faster
  • 🎯 Precision: Transitive dependency tracking ensures no missed tests
  • 🔄 Stateless: No git required → works in any environment (containers, CI, local)
  • 📊 Predictable: Manifest-based → deterministic test selection

Technical Design:

  • Mutually exclusive with --changed (enforced at config resolution)
  • Honors forceRerunTriggers (config changes → full suite)
  • Watch mode compatible (seeds initial run, then watch takes over)
  • Integrates cleanly with existing Vite module graph
  • Manifest stored in Vitest cache directory (node_modules/.vite/vitest/)

Example - Agentic Workflow:

# Agent makes a change to src/utils/parser.ts
vitest run --stale
# ✓ Only runs tests that import parser.ts (directly or transitively)
# ✓ ~10x faster than full suite
# ✓ Agent gets immediate feedback on whether change broke anything

# Agent makes another change
vitest run --stale
# ✓ Only new affected tests run
# ✓ Agent continues rapid iteration

gojimotoro and others added 7 commits March 19, 2026 17:39
Add stale option definition to CLI config, type definition to UserConfig interface, and mutual exclusion check with --changed in resolveConfig.

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Create StaleManifest class for persistent mtime tracking, wire into VitestCache, and initialize in core.ts during setup.

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add filtering logic to detect stale tests based on mtime comparison, handle first-run behavior, and update manifest with scanned files.

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add stale config checks to FilesNotFoundError guard, manifest persistence, watch mode reset blocks, and user messaging.

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add 6 comprehensive test cases covering first-run, subsequent-run, dependency changes, mutual exclusion, and test file changes. Includes fixture directory with test and source files.

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add manual prose section to cli.md explaining first-run behavior, mutual exclusion with --changed, and forceRerunTriggers interaction. Auto-generated cli-generated.md entry.

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add .sisyphus directory to eslint ignores to prevent parsing errors on markdown documentation files.

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
@netlify
Copy link
Copy Markdown

netlify bot commented Mar 19, 2026

Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit f43c300
🔍 Latest deploy log https://app.netlify.com/projects/vitest-dev/deploys/69bc2c39890d750008925726
😎 Deploy Preview https://deploy-preview-9916--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copy link
Copy Markdown
Member

@AriPerkkio AriPerkkio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's first see if custom VCS provider approach would be better solution. It's good to post issue first in feature requests that introduce new changes like this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add --stale CLI option for agentic coding systems

2 participants