Merge pull request #8434 from evildmp/evolutionary-documentation-restructure

Author: nicoddemus

AUTHORS

@@ -78,6 +78,7 @@ Daniel Grana
 Daniel Hahler
 Daniel Nuri
 Daniel Wandschneider
+Daniele Procida
 Danielle Jenkins
 Daniil Galiev
 Dave Hunt

doc/en/_templates/globaltoc.html

@@ -2,12 +2,20 @@ <h3><a href="{{ pathto(master_doc) }}">{{ _('Table Of Contents') }}</a></h3>
 
 <ul>
   <li><a href="{{ pathto('index') }}">Home</a></li>
+
   <li><a href="{{ pathto('getting-started') }}">Get started</a></li>
+  <li><a href="{{ pathto('how-to/index') }}">How-to guides</a></li>
+  <li><a href="{{ pathto('reference/index') }}">Reference guides</a></li>
+  <li><a href="{{ pathto('explanation/index') }}">Explanation</a></li>
+
   <li><a href="{{ pathto('example/index') }}">Examples</a></li>
+
   <li><a href="{{ pathto('reference/customize') }}">Customize</a></li>
   <li><a href="{{ pathto('reference/reference') }}">API Reference</a></li>
   <li><a href="{{ pathto('reference/plugin_list') }}">3rd party plugins</a></li>
+
   <li><a href="{{ pathto('contents') }}">Complete table of contents</a></li>
+
   <li><a href="{{ pathto('changelog') }}">Changelog</a></li>
   <li><a href="{{ pathto('contributing') }}">Contributing</a></li>
   <li><a href="{{ pathto('backwards-compatibility') }}">Backwards Compatibility</a></li>

doc/en/contents.rst

@@ -35,6 +35,7 @@ How-to guides
    how-to/plugins
    how-to/nose
    how-to/bash-completion
+   how-to/fixtures
 
 
 Reference guides
@@ -43,7 +44,7 @@ Reference guides
 .. toctree::
    :maxdepth: 2
 
-   reference/fixture
+   reference/fixtures
    reference/warnings
    reference/doctest
    reference/cache
@@ -56,15 +57,25 @@ Reference guides
    reference/reference
 
 
+Explanation
+-----------------
+
+.. toctree::
+   :maxdepth: 2
+
+   explanation/anatomy
+   explanation/fixtures
+   explanation/goodpractices
+   explanation/flaky
+   explanation/pythonpath
+
+
 Further topics
 -----------------
 
 .. toctree::
    :maxdepth: 2
 
-   goodpractices
-   flaky
-   pythonpath
    example/index
 
    backwards-compatibility

doc/en/example/simple.rst

@@ -69,7 +69,7 @@ Here is a basic pattern to achieve this:
 
 
 For this to work we need to add a command line option and
-provide the ``cmdopt`` through a :ref:`fixture function <fixture function>`:
+provide the ``cmdopt`` through a :ref:`fixture function <fixture>`:
 
 .. code-block:: python
 

doc/en/explanation/anatomy.rst

@@ -0,0 +1,46 @@
+.. _test-anatomy:
+
+Anatomy of a test
+=================
+
+In the simplest terms, a test is meant to look at the result of a particular
+behavior, and make sure that result aligns with what you would expect.
+Behavior is not something that can be empirically measured, which is why writing
+tests can be challenging.
+
+"Behavior" is the way in which some system **acts in response** to a particular
+situation and/or stimuli. But exactly *how* or *why* something is done is not
+quite as important as *what* was done.
+
+You can think of a test as being broken down into four steps:
+
+1. **Arrange**
+2. **Act**
+3. **Assert**
+4. **Cleanup**
+
+**Arrange** is where we prepare everything for our test. This means pretty
+much everything except for the "**act**". It's lining up the dominoes so that
+the **act** can do its thing in one, state-changing step. This can mean
+preparing objects, starting/killing services, entering records into a database,
+or even things like defining a URL to query, generating some credentials for a
+user that doesn't exist yet, or just waiting for some process to finish.
+
+**Act** is the singular, state-changing action that kicks off the **behavior**
+we want to test. This behavior is what carries out the changing of the state of
+the system under test (SUT), and it's the resulting changed state that we can
+look at to make a judgement about the behavior. This typically takes the form of
+a function/method call.
+
+**Assert** is where we look at that resulting state and check if it looks how
+we'd expect after the dust has settled. It's where we gather evidence to say the
+behavior does or does not aligns with what we expect. The ``assert`` in our test
+is where we take that measurement/observation and apply our judgement to it. If
+something should be green, we'd say ``assert thing == "green"``.
+
+**Cleanup** is where the test picks up after itself, so other tests aren't being
+accidentally influenced by it.
+
+At its core, the test is ultimately the **act** and **assert** steps, with the
+**arrange** step only providing the context. **Behavior** exists between **act**
+and **assert**.

doc/en/explanation/fixtures.rst

@@ -0,0 +1,157 @@
+.. _about-fixtures:
+
+About fixtures
+===============
+
+.. seealso:: :ref:`how-to-fixtures`
+.. seealso:: :ref:`Fixtures reference <reference-fixtures>`
+
+
+What fixtures are
+-----------------
+
+In testing, a `fixture <https://en.wikipedia.org/wiki/Test_fixture#Software>`_
+provides a defined, reliable and consistent context for the tests. This could
+include environment (for example a database configured with known parameters)
+or content (such as a dataset).
+
+Fixtures define the steps and data that constitute the *arrange* phase of a
+test (see :ref:`test-anatomy`). In pytest, they are functions you define that
+serve this purpose. They can also be used to define a test's *act* phase; this
+is a powerful technique for designing more complex tests.
+
+The services, state, or other operating environments set up by fixtures are
+accessed by test functions through arguments. For each fixture used by a test
+function there is typically a parameter (named after the fixture) in the test
+function's definition.
+
+We can tell pytest that a particular function is a fixture by decorating it with
+:py:func:`@pytest.fixture <pytest.fixture>`. Here's a simple example of
+what a fixture in pytest might look like:
+
+.. code-block:: python
+
+    import pytest
+
+
+    class Fruit:
+        def __init__(self, name):
+            self.name = name
+
+        def __eq__(self, other):
+            return self.name == other.name
+
+
+    @pytest.fixture
+    def my_fruit():
+        return Fruit("apple")
+
+
+    @pytest.fixture
+    def fruit_basket(my_fruit):
+        return [Fruit("banana"), my_fruit]
+
+
+    def test_my_fruit_in_basket(my_fruit, fruit_basket):
+        assert my_fruit in fruit_basket
+
+Tests don't have to be limited to a single fixture, either. They can depend on
+as many fixtures as you want, and fixtures can use other fixtures, as well. This
+is where pytest's fixture system really shines.
+
+
+Improvements over xUnit-style setup/teardown functions
+-----------------------------------------------------------
+
+pytest fixtures offer dramatic improvements over the classic xUnit
+style of setup/teardown functions:
+
+* fixtures have explicit names and are activated by declaring their use
+  from test functions, modules, classes or whole projects.
+
+* fixtures are implemented in a modular manner, as each fixture name
+  triggers a *fixture function* which can itself use other fixtures.
+
+* fixture management scales from simple unit to complex
+  functional testing, allowing to parametrize fixtures and tests according
+  to configuration and component options, or to re-use fixtures
+  across function, class, module or whole test session scopes.
+
+* teardown logic can be easily, and safely managed, no matter how many fixtures
+  are used, without the need to carefully handle errors by hand or micromanage
+  the order that cleanup steps are added.
+
+In addition, pytest continues to support :ref:`xunitsetup`.  You can mix
+both styles, moving incrementally from classic to new style, as you
+prefer.  You can also start out from existing :ref:`unittest.TestCase
+style <unittest.TestCase>` or :ref:`nose based <nosestyle>` projects.
+
+
+
+Fixture errors
+--------------
+
+pytest does its best to put all the fixtures for a given test in a linear order
+so that it can see which fixture happens first, second, third, and so on. If an
+earlier fixture has a problem, though, and raises an exception, pytest will stop
+executing fixtures for that test and mark the test as having an error.
+
+When a test is marked as having an error, it doesn't mean the test failed,
+though. It just means the test couldn't even be attempted because one of the
+things it depends on had a problem.
+
+This is one reason why it's a good idea to cut out as many unnecessary
+dependencies as possible for a given test. That way a problem in something
+unrelated isn't causing us to have an incomplete picture of what may or may not
+have issues.
+
+Here's a quick example to help explain:
+
+.. code-block:: python
+
+    import pytest
+
+
+    @pytest.fixture
+    def order():
+        return []
+
+
+    @pytest.fixture
+    def append_first(order):
+        order.append(1)
+
+
+    @pytest.fixture
+    def append_second(order, append_first):
+        order.extend([2])
+
+
+    @pytest.fixture(autouse=True)
+    def append_third(order, append_second):
+        order += [3]
+
+
+    def test_order(order):
+        assert order == [1, 2, 3]
+
+
+If, for whatever reason, ``order.append(1)`` had a bug and it raises an exception,
+we wouldn't be able to know if ``order.extend([2])`` or ``order += [3]`` would
+also have problems. After ``append_first`` throws an exception, pytest won't run
+any more fixtures for ``test_order``, and it won't even try to run
+``test_order`` itself. The only things that would've run would be ``order`` and
+``append_first``.
+
+
+Sharing test data
+-----------------
+
+If you want to make test data from files available to your tests, a good way
+to do this is by loading these data in a fixture for use by your tests.
+This makes use of the automatic caching mechanisms of pytest.
+
+Another good approach is by adding the data files in the ``tests`` folder.
+There are also community plugins available to help managing this aspect of
+testing, e.g. `pytest-datadir <https://pypi.org/project/pytest-datadir/>`__
+and `pytest-datafiles <https://pypi.org/project/pytest-datafiles/>`__.

doc/en/flaky.rst renamed to doc/en/explanation/flaky.rst

@@ -48,7 +48,7 @@ Conventions for Python test discovery
   * ``test`` prefixed test functions or methods outside of class
   * ``test`` prefixed test functions or methods inside ``Test`` prefixed test classes (without an ``__init__`` method)
 
-For examples of how to customize your test discovery :doc:`example/pythoncollection`.
+For examples of how to customize your test discovery :doc:`/example/pythoncollection`.
 
 Within Python modules, ``pytest`` also discovers tests using the standard
 :ref:`unittest.TestCase <unittest.TestCase>` subclassing technique.

doc/en/goodpractices.rst renamed to doc/en/explanation/goodpractices.rst

@@ -0,0 +1,13 @@
+:orphan:
+
+Explanation
+================
+
+.. toctree::
+   :maxdepth: 1
+
+   anatomy
+   fixtures
+   goodpractices
+   flaky
+   pythonpath