Test Coverage with Istanbul

[Visible-Radio]

I am aware of this previous discussion: #2284

However it seems like things have changed since then.

Since Remix has embraced vite, I was hoping vite-plugin-istanbul could be used to instrument a remix app in order to get coverage results when testing with playwright.

I can't find any documentation anywhere to do such a thing. Is there still some inherent reason why this won't work?

[itxch]

This is really interesting, I didn't know that was actually a thing - have you tried it / what errors/output dyou get?

[Visible-Radio]

@itxch

I'm not sure that it is a thing - but folks seem to be doing it with Next.js using babel-plugin-istanbul

https://dev.to/anishkny/code-coverage-for-a-nextjs-app-using-playwright-tests-18n7

And with vite-built apps using vite-plugin-istanbul

https://dev.to/giannispapadakis/code-coverage-with-react-vite-rtl-cypress-12h0

There is this package that includes some playwright fixtures for collecting coverage, also using vite-plugin-istanbul

https://github.com/mxschmitt/playwright-test-coverage

I haven't had success instrumenting remix with it however. I am actually getting some kind of coverage results - but it's not accurate; code that I know is exercised by my tests shows 0 lines covered, and not all files are reported, etc.

I am no build tool expert, so I could be missing something obvious. Between the configuration for remix, nyc, and the vite istanbul plugin, I'm really at a loss as to hook it up correctly.

[spencersteers]

I was able to get coverage for client side code using https://github.com/mxschmitt/playwright-test-coverage.

No luck with anything server side though (loaders, actions, ect).

[bgfernandes]

I managed to get server side coverage, but I still need to look into setting up coverage for the front-end code that are served from the public/build folder. Thanks for the link, I will check out if that works for our setup.

[bgfernandes]

I was able to get back-end coverage using Istanbul's nyc https://github.com/istanbuljs/nyc

Basically, I run the whole playwright wrapped by nyc like this: yarn nyc --reporter=lcov playwright test.

I'm not a remix expert, I don't work directly with the team that builds that particular app, but they have an odd setup, their entry server.js file is importing the built remix bundle, for the source maps to be interpreted correctly I needed to specify both source and build folders as entries to istanbul.

Here is my .nycrc file:

{
  "include": [
    "src/**",
    "build/**",
    "server.js"
  ]
}