Skip to content

Fixed pytest reporting to correctly track xfail/xpass#133

Open
aveeyan wants to merge 2 commits intomainfrom
fix/conftest
Open

Fixed pytest reporting to correctly track xfail/xpass#133
aveeyan wants to merge 2 commits intomainfrom
fix/conftest

Conversation

@aveeyan
Copy link
Collaborator

@aveeyan aveeyan commented Oct 13, 2025

Fix pytest xfailed/xpassed tracking and pass percentage calculation

Problem

  • Xfailed and xpassed test counters always showed 0
  • Pass percentage calculation was incorrect (denominator didn't exclude xfailed tests)
  • Tests were being double-counted in some scenarios

Solution

  • Switched from pytest_runtest_makereport to pytest_runtest_logreport hook for reliable outcome detection
  • Fixed denominator: total - skipped - xfailed (previously was total - skipped)
  • Added deduplication to prevent counting tests multiple times

Verification

Tested against test files - all counts now match pytest's native output:

  • test_html_utils.py: 49 total, 14 passed, 19 skipped, 3 xfailed, 13 xpassed ✓
  • test_debug.py: 43 total, 19 passed, 0 skipped, 2 xfailed, 22 xpassed ✓

Changes

  • Enhanced output to show denominator breakdown for transparency
  • Pass percentage now correctly excludes both skipped and xfailed tests

…pytest_runtest_makereport to pytest_runtest_logreport for proper outcome detection
@aveeyan aveeyan requested a review from tomasohara as a code owner October 13, 2025 17:25
@tomasohara
Copy link
Owner

tomasohara commented Oct 13, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants