Inconsistent wasxfail value when combining pytest.mark.xfail fixture with explicit calls to pytest.xfail or pytest.fail #14090
Description
We get inconsistent wasxfail values when combining pytest.mark.xfail fixture with explicit calls to pytest.xfail or pytest.fail.
The mark.xfail reason gets ignored in the presence of those explicit calls.
Use the following conftest.py file
import pytest
@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
outcome = yield
report = outcome.get_result()
if call.when == "call":
wasxfail = getattr(report, "wasxfail", None)
msg = None
if hasattr(call.excinfo, "value") and hasattr(call.excinfo.value, "msg"):
msg = call.excinfo.value.msg
print("\ntest: ", item.name)
print("outcome: ", report.outcome)
print("msg: ", msg)
print("wasxfail: ", wasxfail)
Use the following test file
import pytest
@pytest.mark.xfail(reason="reason mark.xfail")
def test_func1():
pytest.fail("reason pytest.fail")
@pytest.mark.xfail(reason="reason mark.xfail")
def test_func2():
pytest.fail()
@pytest.mark.xfail(reason="reason mark.xfail")
def test_func3():
pytest.xfail("reason pytest.xfail")
@pytest.mark.xfail(reason="reason mark.xfail")
def test_func4():
pytest.xfail()
Result:
test: test_func1
outcome: skipped
msg: reason pytest.fail
wasxfail: reason pytest.fail => should be "reason mark.xfail" ??
test: test_func2
outcome: skipped
msg:
wasxfail: => empty string. why? should be "reason mark.xfail"
test: test_func3
outcome: skipped
msg: reason pytest.xfail
wasxfail: reason pytest.xfail => should be "reason mark.xfail" ??
test: test_func4
outcome: skipped
msg:
wasxfail: => empty string. why? should be "reason mark.xfail"
See PR #14091