Improve pytest.raises docs (#11578)

Author: nicoddemus

doc/en/how-to/assert.rst

@@ -98,6 +98,27 @@ and if you need to have access to the actual exception info you may use:
 the actual exception raised.  The main attributes of interest are
 ``.type``, ``.value`` and ``.traceback``.
 
+Note that ``pytest.raises`` will match the exception type or any subclasses (like the standard ``except`` statement).
+If you want to check if a block of code is raising an exact exception type, you need to check that explicitly:
+
+
+.. code-block:: python
+
+    def test_recursion_depth():
+        def foo():
+            raise NotImplementedError
+
+        with pytest.raises(RuntimeError) as excinfo:
+            foo()
+        assert type(excinfo.value) is RuntimeError
+
+The :func:`pytest.raises` call will succeed, even though the function raises :class:`NotImplementedError`, because
+:class:`NotImplementedError` is a subclass of :class:`RuntimeError`; however the following `assert` statement will
+catch the problem.
+
+Matching exception messages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 You can pass a ``match`` keyword parameter to the context-manager to test
 that a regular expression matches on the string representation of an exception
 (similar to the ``TestCase.assertRaisesRegex`` method from ``unittest``):
@@ -115,9 +136,15 @@ that a regular expression matches on the string representation of an exception
         with pytest.raises(ValueError, match=r".* 123 .*"):
             myfunc()
 
-The regexp parameter of the ``match`` parameter is matched with the ``re.search``
-function, so in the above example ``match='123'`` would have worked as
-well.
+Notes:
+
+* The ``match`` parameter is matched with the :func:`re.search`
+  function, so in the above example ``match='123'`` would have worked as well.
+* The ``match`` parameter also matches against `PEP-678 <https://peps.python.org/pep-0678/>`__ ``__notes__``.
+
+
+Matching exception groups
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
 You can also use the :func:`excinfo.group_contains() <pytest.ExceptionInfo.group_contains>`
 method to test for exceptions returned as part of an ``ExceptionGroup``:
@@ -165,32 +192,55 @@ exception at a specific level; exceptions contained directly in the top
         assert not excinfo.group_contains(RuntimeError, depth=2)
         assert not excinfo.group_contains(TypeError, depth=1)
 
-There's an alternate form of the :func:`pytest.raises` function where you pass
-a function that will be executed with the given ``*args`` and ``**kwargs`` and
-assert that the given exception is raised:
+Alternate form (legacy)
+~~~~~~~~~~~~~~~~~~~~~~~
+
+There is an alternate form where you pass
+a function that will be executed, along ``*args`` and ``**kwargs``, and :func:`pytest.raises`
+will execute the function with the arguments and assert that the given exception is raised:
 
 .. code-block:: python
 
-    pytest.raises(ExpectedException, func, *args, **kwargs)
+    def func(x):
+        if x <= 0:
+            raise ValueError("x needs to be larger than zero")
+
+
+    pytest.raises(ValueError, func, x=-1)
 
 The reporter will provide you with helpful output in case of failures such as *no
 exception* or *wrong exception*.
 
-Note that it is also possible to specify a "raises" argument to
-``pytest.mark.xfail``, which checks that the test is failing in a more
+This form was the original :func:`pytest.raises` API, developed before the ``with`` statement was
+added to the Python language. Nowadays, this form is rarely used, with the context-manager form (using ``with``)
+being considered more readable.
+Nonetheless, this form is fully supported and not deprecated in any way.
+
+xfail mark and pytest.raises
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It is also possible to specify a ``raises`` argument to
+:ref:`pytest.mark.xfail <pytest.mark.xfail ref>`, which checks that the test is failing in a more
 specific way than just having any exception raised:
 
 .. code-block:: python
 
+    def f():
+        raise IndexError()
+
+
     @pytest.mark.xfail(raises=IndexError)
     def test_f():
         f()
 
-Using :func:`pytest.raises` is likely to be better for cases where you are
-testing exceptions your own code is deliberately raising, whereas using
-``@pytest.mark.xfail`` with a check function is probably better for something
-like documenting unfixed bugs (where the test describes what "should" happen)
-or bugs in dependencies.
+
+This will only "xfail" if the test fails by raising ``IndexError`` or subclasses.
+
+* Using :ref:`pytest.mark.xfail <pytest.mark.xfail ref>` with the ``raises`` parameter is probably better for something
+  like documenting unfixed bugs (where the test describes what "should" happen) or bugs in dependencies.
+
+* Using :func:`pytest.raises` is likely to be better for cases where you are
+  testing exceptions your own code is deliberately raising, which is the majority of cases.
 
 
 .. _`assertwarns`:

doc/en/reference/reference.rst

@@ -249,7 +249,9 @@ Marks a test function as *expected to fail*.
     :keyword str reason:
         Reason why the test function is marked as xfail.
     :keyword Type[Exception] raises:
-        Exception subclass (or tuple of subclasses) expected to be raised by the test function; other exceptions will fail the test.
+        Exception class (or tuple of classes) expected to be raised by the test function; other exceptions will fail the test.
+        Note that subclasses of the classes passed will also result in a match (similar to how the ``except`` statement works).
+
     :keyword bool run:
         Whether the test function should actually be executed. If ``False``, the function will always xfail and will
         not be executed (useful if a function is segfaulting).

src/_pytest/python_api.py

@@ -804,11 +804,13 @@ def raises(  # noqa: F811
 def raises(  # noqa: F811
     expected_exception: Union[Type[E], Tuple[Type[E], ...]], *args: Any, **kwargs: Any
 ) -> Union["RaisesContext[E]", _pytest._code.ExceptionInfo[E]]:
-    r"""Assert that a code block/function call raises an exception.
+    r"""Assert that a code block/function call raises an exception type, or one of its subclasses.
 
     :param typing.Type[E] | typing.Tuple[typing.Type[E], ...] expected_exception:
         The expected exception type, or a tuple if one of multiple possible
-        exception types are expected.
+        exception types are expected. Note that subclasses of the passed exceptions
+        will also match.
+
     :kwparam str | typing.Pattern[str] | None match:
         If specified, a string containing a regular expression,
         or a regular expression object, that is tested against the string
@@ -826,13 +828,13 @@ def raises(  # noqa: F811
     .. currentmodule:: _pytest._code
 
     Use ``pytest.raises`` as a context manager, which will capture the exception of the given
-    type::
+    type, or any of its subclasses::
 
         >>> import pytest
         >>> with pytest.raises(ZeroDivisionError):
         ...    1/0
 
-    If the code block does not raise the expected exception (``ZeroDivisionError`` in the example
+    If the code block does not raise the expected exception (:class:`ZeroDivisionError` in the example
     above), or no exception at all, the check will fail instead.
 
     You can also use the keyword argument ``match`` to assert that the
@@ -845,7 +847,7 @@ def raises(  # noqa: F811
         ...     raise ValueError("value must be 42")
 
     The ``match`` argument searches the formatted exception string, which includes any
-    `PEP-678 <https://peps.python.org/pep-0678/>` ``__notes__``:
+    `PEP-678 <https://peps.python.org/pep-0678/>`__ ``__notes__``:
 
         >>> with pytest.raises(ValueError, match=r'had a note added'):  # doctest: +SKIP
         ...    e = ValueError("value must be 42")
@@ -860,6 +862,20 @@ def raises(  # noqa: F811
         >>> assert exc_info.type is ValueError
         >>> assert exc_info.value.args[0] == "value must be 42"
 
+    .. warning::
+
+       Given that ``pytest.raises`` matches subclasses, be wary of using it to match :class:`Exception` like this::
+
+           with pytest.raises(Exception):  # Careful, this will catch ANY exception raised.
+                some_function()
+
+       Because :class:`Exception` is the base class of almost all exceptions, it is easy for this to hide
+       real bugs, where the user wrote this expecting a specific exception, but some other exception is being
+       raised due to a bug introduced during a refactoring.
+
+       Avoid using ``pytest.raises`` to catch :class:`Exception` unless certain that you really want to catch
+       **any** exception raised.
+
     .. note::
 
        When using ``pytest.raises`` as a context manager, it's worthwhile to
@@ -872,7 +888,7 @@ def raises(  # noqa: F811
            >>> with pytest.raises(ValueError) as exc_info:
            ...     if value > 10:
            ...         raise ValueError("value must be <= 10")
-           ...     assert exc_info.type is ValueError  # this will not execute
+           ...     assert exc_info.type is ValueError  # This will not execute.
 
        Instead, the following approach must be taken (note the difference in
        scope)::
@@ -891,6 +907,10 @@ def raises(  # noqa: F811
 
     See :ref:`parametrizing_conditional_raising` for an example.
 
+    .. seealso::
+
+        :ref:`assertraises` for more examples and detailed discussion.
+
     **Legacy form**
 
     It is possible to specify a callable by passing a to-be-called lambda::