Skip to content

feat(coverage): default to text reporter skipFull if agent detected#10018

Open
hi-ogawa wants to merge 4 commits intovitest-dev:mainfrom
hi-ogawa:feat-coverage-default-to-text-reporter-skipfull-if-agent-detected
Open

feat(coverage): default to text reporter skipFull if agent detected#10018
hi-ogawa wants to merge 4 commits intovitest-dev:mainfrom
hi-ogawa:feat-coverage-default-to-text-reporter-skipfull-if-agent-detected

Conversation

@hi-ogawa
Copy link
Copy Markdown
Collaborator

@hi-ogawa hi-ogawa commented Mar 29, 2026

Description

I was prototyping a new text-agent coverage reporter, which skips 100% coverage just like test agent reporter (#9779) skips passing tests. But then I noticed text coverage reporter already supports skipFull which does mostly same.

This PR auto enables skipFull when the agent is detected.
(EDIT: and also auto configure text-summary, so it has aggregated stats to avoid ended up with empty output.)

  • without agent
~/code/others/vitest $ pnpm -C examples/fastify test --coverage 
...
 DEV  v4.1.2 /home/hiroshi/code/others/vitest/examples/fastify
      Coverage enabled with v8

 ✓ test/app.test.ts (3 tests) 55ms
   ✓ with HTTP injection 15ms
   ✓ with a running server 13ms
   ✓ with axios 25ms

 Test Files  1 passed (1)
      Tests  3 passed (3)
   Start at  19:18:31
   Duration  465ms (transform 39ms, setup 0ms, import 294ms, tests 55ms, environment 0ms)

 % Coverage report from v8
--------------|---------|----------|---------|---------|-------------------
File          | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
--------------|---------|----------|---------|---------|-------------------
All files     |     100 |      100 |     100 |     100 |                   
 fastify      |     100 |      100 |     100 |     100 |                   
  mockData.ts |     100 |      100 |     100 |     100 |                   
 fastify/src  |     100 |      100 |     100 |     100 |                   
  app.ts      |     100 |      100 |     100 |     100 |                   
--------------|---------|----------|---------|---------|-------------------
 PASS  Waiting for file changes...
       press h to show help, press q to quit
  • with agent
~/code/others/vitest $ AI_AGENT=true pnpm -C examples/fastify test --coverage 
...
 RUN  v4.1.2 /home/hiroshi/code/others/vitest/examples/fastify
      Coverage enabled with v8


 Test Files  1 passed (1)
      Tests  3 passed (3)
   Start at  19:18:37
   Duration  295ms (transform 35ms, setup 0ms, import 129ms, tests 49ms, environment 0ms)

 % Coverage report from v8
--------------|---------|----------|---------|---------|-------------------
File          | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
--------------|---------|----------|---------|---------|-------------------
--------------|---------|----------|---------|---------|-------------------
=============================== Coverage summary ===============================
Statements   : 100% ( 4/4 )
Branches     : 100% ( 0/0 )
Functions    : 100% ( 1/1 )
Lines        : 100% ( 4/4 )
================================================================================

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.
  • Please check Allow edits by maintainers to make review process faster. Note that this option is not available for repositories that are owned by Github organizations.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

@netlify
Copy link
Copy Markdown

netlify bot commented Mar 29, 2026

Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit a5c563a
🔍 Latest deploy log https://app.netlify.com/projects/vitest-dev/deploys/69c9c7d71dfd940008991b21
😎 Deploy Preview https://deploy-preview-10018--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@hi-ogawa hi-ogawa marked this pull request as ready for review March 29, 2026 11:01
AriPerkkio
AriPerkkio previously approved these changes Mar 29, 2026
Copy link
Copy Markdown
Member

@AriPerkkio AriPerkkio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea but I wonder if it confuses agents or humans to see empty reports.

text reporter should probably print something like ... and 50 files with 100% coverage. 🤔

@hi-ogawa
Copy link
Copy Markdown
Collaborator Author

Good idea but I wonder if it confuses agents or humans to see empty reports.

text reporter should probably print something like ... and 50 files with 100% coverage. 🤔

We can also ensure text-summary at the same time, but this still doesn't show file stats. Any customization beyond this requires Vitest's own builtin coverage reporters.

 % Coverage report from v8
--------------|---------|----------|---------|---------|-------------------
File          | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
--------------|---------|----------|---------|---------|-------------------
--------------|---------|----------|---------|---------|-------------------

=============================== Coverage summary ===============================
Statements   : 100% ( 4/4 )
Branches     : 100% ( 0/0 )
Functions    : 100% ( 1/1 )
Lines        : 100% ( 4/4 )
================================================================================

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants