Skip to content

Quick modes pytest-cov results are misleading #1

@gvee-uk

Description

@gvee-uk

Should the pre-commit fail, you get loads of output in the terminal which includes the output from pytest-cov/coverage.

If you run the pre-commit hook without the --quick argument, this output is misleading.

For an example, here's the output from a repository that has significant coverage, when running in the slower mode the output:

Name                                                                      Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------------------------------------
...
-------------------------------------------------------------------------------------------------------
TOTAL                                                                     12277    135    99%

And then when running in --quick mode:

Name                                                                      Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------------------------------------
...
-------------------------------------------------------------------------------------------------------
TOTAL                                                                     12277   5106    58%

This output is very misleading to the end user, as they probably aren't expecting such a low coverage percentage!

It looks like one-or-more of the arguments appended to pytest[-cov] when running in quick mode skips a bunch of tests, causing the number of misses to increase (which makes logical sense when looking at the args).

I wonder if the output from pytest should be supressed in all instances? What happens if you have a failing test in your suite then?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions