Skip to content

feat: support merge reports for non-sharded multi-environment runs#9967

Open
hi-ogawa wants to merge 46 commits intovitest-dev:mainfrom
hi-ogawa:feat/merge-reports-label
Open

feat: support merge reports for non-sharded multi-environment runs#9967
hi-ogawa wants to merge 46 commits intovitest-dev:mainfrom
hi-ogawa:feat/merge-reports-label

Conversation

@hi-ogawa
Copy link
Copy Markdown
Collaborator

@hi-ogawa hi-ogawa commented Mar 25, 2026

Description

Example

Set VITEST_BLOB_LABEL environment variable (or [["blob", { label }]] in config) when creating blobs:

VITEST_BLOB_LABEL=linux vitest run --reporter=blob
VITEST_BLOB_LABEL=macos vitest run --reporter=blob
// Or in config
export default defineConfig({
  test: {
    reporters: [['blob', { label: 'linux' }]],
  },
})

The reporter during --merge-reports can show the test run results from different blob label as if they are from different projects:

$ vitest run --merge-reports
...
 ✓  linux  src/basic.test.ts (2 tests)
 ✓  macos  src/basic.test.ts (2 tests)

How it works

The label is a blob reporter option — it does not affect the test run itself.

Write side: The blob reporter stores the label in the blob payload. The label also determines the default output filename (blob-linux.json). Project names are not mutated during the test run.

Read side: readBlobs() discovers labels from parsed blob content, clones TestProject instances per label (same pattern as browser mode), then rewrites file.projectName, File.id, child task IDs, and environmentModules keys before module graph restoration. The merge pipeline downstream sees properly-discriminated projects with no collisions.

TODO

🎦 Examples from dog-fooding

  • test/cli
image
  • test/core
image

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.
  • Please check Allow edits by maintainers to make review process faster. Note that this option is not available for repositories that are owned by Github organizations.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

@netlify
Copy link
Copy Markdown

netlify bot commented Mar 25, 2026

Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit 5a4e63d
🔍 Latest deploy log https://app.netlify.com/projects/vitest-dev/deploys/69cb2ced83026f0008982546
😎 Deploy Preview https://deploy-preview-9967--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

hi-ogawa and others added 8 commits March 25, 2026 11:59
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… label

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@hi-ogawa hi-ogawa changed the title feat: support reports for non-sharded multi-environment runs feat: support merge reports for non-sharded multi-environment runs Mar 25, 2026
hi-ogawa and others added 3 commits March 25, 2026 12:36
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The unit test with vi.mock for node:os breaks in test/cli (isolate: false)
when other tests cause node:os to be cached before the mock applies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@hi-ogawa hi-ogawa marked this pull request as ready for review March 25, 2026 05:15
import { describe, expect, test, vi } from 'vitest'
import { getWorkersCountByPercentage } from 'vitest/src/utils/workers.js'

vi.mock(import('node:os'), async importOriginal => ({
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unit test is evacuated to test/core since vi.mock inside test/cli is breaking something.

Copy link
Copy Markdown
Member

@sheremet-va sheremet-va left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am confused by the implementation. Wouldn't it make more sense to change the project's name when running tests so they are collected with new ids instead of patching them after?

@hi-ogawa
Copy link
Copy Markdown
Collaborator Author

I am confused by the implementation. Wouldn't it make more sense to change the project's name when running tests so they are collected with new ids instead of patching them after?

I basically prototype both but I preferred current version since Blob reporter option influencing test run seems wrong and merge-time patch didn't look too far fetched.
For example, if user test code checks project name during test run to do something specifically, then that code needs to take into account blob labeling. The same can be true for patching-in project name for merge time reporter though. The choice is just based on my initial trade-off instinct. We should discuss to figure out which is better.

@sheremet-va
Copy link
Copy Markdown
Member

We should discuss to figure out which is better.

In my opinion, it's better to change the name when running tests to reduce the complexity. I would like to avoid regenerating and recalculating hashes.

As an option, we could add a static field that influences the hash (like file.hashKey or something). I would also be useful for typechecking (which injects __typecheck__ into a file id right now)

@hi-ogawa
Copy link
Copy Markdown
Collaborator Author

hi-ogawa commented Mar 27, 2026

Based on the discussion, I'm going to explore the approach to encode "blob label" into File.id and File.meta instead of differentiating the test files by artificially created "label project". The reporter makes use of File.meta to differentiate visually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support merging reports for non-sharded multi-environment runs

2 participants