Test coverage in Next.js codebase

[fabianishere]

Hi,

Is there a particular reason why the test coverage is not being measured in the CI pipelines?

From what I can find, Next.js at one point in time did have support for measuring code coverage (#244 and c56fde89) and also published the results to Coveralls, but this was eventually removed from the repository (#7689). Is there any reason for this?

[timneutkens]

The coverage being tracked was wrong and we didn't use it at the time so we removed it.

Almost all our tests are end-to-end integration tests booting up Next.js and the internal files are bundled making it really hard to track code coverage as-is.

However our test suite is very extensive and is continuously expanded: https://github.com/zeit/next.js/tree/canary/test/integration

[timneutkens]

Well overall the coverage was not useful as far as we used it so it was removed to not confuse users with "fake news" coverage.

[fabianishere]

Pardon my curiosity, but was that due to the coverage being "wrong"? In other words, do you see value in proper test coverage of e2e tests, for example to see whether some code paths might not have been properly covered by the test?

[timneutkens]

The coverage only took into account a small part of the integration tests. Today it would have tracked even less as we changed the E2E tests to boot up Next.js directly.

The confidence in the test suite is very high so test coverage would be a nice to have rather than extremely useful.

In general projects need code coverage as they don't have confidence in their test suite covering the majority of features

[fabianishere]

For context: As part of a course on Software Architecture, I am writing a small series of blog posts on the architecture of Next.js and at the moment I am investigating the test processes that Next.js uses.

[fabianishere]

Agreed! Thank you for the clarification :)