recwarn: warnings are re-emitted with wrong module

[bluetech]

While reviewing #11917 I noticed a problem with the warning re-emitting code added in pytest 8.0. I think the module=w.__module__ line below is wrong:

pytest/src/_pytest/recwarn.py

Lines 324 to 331 in aaa9ca7

	warnings.warn_explicit(
	str(w.message),
	w.message.__class__, # type: ignore[arg-type]
	w.filename,
	w.lineno,
	module=w.__module__,
	source=w.source,
	)

w here is a warnings.WarningMessage so w.__module__ is always "warnings". But the warning.warn_explicit say this should rather be the module of the warning that is used for filtering.

If I'm reading the warnings code correctly, the module originally passed by the user is not preserved, so the warning cannot be re-emitted faithfully in this regard, but we can probably do something better than the current situation.

cc @reaganjlee @Zac-HD

[Zac-HD]

Ah, yeah, we'll need to recover the module name from w.filename and inspecting sys.modules.

[reaganjlee]

Apologies if I'm missing something. The warn_explicit() docs say

The module name defaults to the filename with .py stripped

w.filename lists the full path (e.g. '/path/to/error-file.py'). Is it possible to leave module to this default? Or does the actual module get lost somehow

[Zac-HD]

OK, here's what I think is going on - there are several things with too-similar names and we're getting them mixed up.

• In warn_explicit( ..., module=, ...), the module argument should be the importable name of the module - e.g. "random" or "urllib.parse". These can be found as keys in the sys.modules dict; not to be confused with the module objects which are the values in that dict.
  • Our bug was that w.__module__ is warnings.WarningMessage.__class__.__module__, i.e. always "warnings", rather than the location from which the instance-of-Warning (aka message) was emitted. Oops!
• "The module name defaults to the filename with .py stripped" - my interpretation of this is that /path/to/error_file.py would use "error_file" as the module= argument, though I'm not confident without running some experiments.
• The original module= value doesn't seem to be tracked anywhere - it's used to find the relevant warnings registry, but not stored. I think we can recover it, at least in most cases, with next(k for k, v in sys.modules.items() if getattr(v, "__file__", None) == m.filename), handling the not-found case - or maybe create and cache a filename: modulename dict.

So it's going to be imperfect, but we can still improve somewhat on the status quo 🙂

[aaronzo]

Ran into this today - I have a (potentially common) setup where:

• I don't want pytest to show me 'transitive' warnings from 3rd party libs I can't change, so I have:

  [tool.pytest.ini_options]
  filterwarnings = [
      'ignore:::<3rd-party-lib>',
  ]

  in my config

• My library sometimes raises warnings (from stdlib), and I want to test that in unit tests with:

   with pytest.warns(UserWarning, ...):
        ...  # code here doesn't respect the module filter

@Zac-HD 's solution seems sensible but possibly expensive? I'd like to know in what scenarios that wouldn't work.

I'll propose an alternative solution/workaround:

Add a kwarg to pytest.warns called keep_ignores which defaults to False (for backwards compatibility). When set to True pytest does not reset filters on this line:

pytest/src/_pytest/recwarn.py

Line 236 in 9cc6b50

warnings.simplefilter("always")

[aaronzo]

I've added a #12897 implementing my idea. Any reviews welcome!

[reaganjlee]

Added workaround comment to PR (tldr probably just better to make the change on our end than adding user-facing arg) -- For the cost point, maybe it could be easier to just create an InternalSuspendedWarning representation with the module data directly? (If it can be conveniently done that way 😁, I can check soon) This doesn't make much sense haha

I did get to do a benchmark of Zac's solution though and it goes from 1.136 to 1.536 sec summed 10,000 examples so doesn't seem too bad