Ignore ordering if it would break a dependency

- add configuration option for ordering all dependencies defined by the
  pytest-dependency plugin
- fix typo in sparse ordering implementation, add test

Author: mrbean-bremen

CHANGELOG.md

@@ -6,8 +6,16 @@
 - added configuration option for sparse sorting, e.g. the possibility to
   fill gaps between ordinals with unordered tests (see also 
   [this issue](https://github.com/ftobia/pytest-ordering/issues/14) in
-  `pytest-ordering`)
-
+  `pytest-ordering`) 
+- ignore ordering if it would break a dependency defined by the
+  `pytest-dependency` plugin
+- experimental: added configuration option for ordering all dependencies
+  defined by the `pytest-dependency` plugin
+  
+### Infrastructure
+- added list of open issues in `pytest-ordering` with respective state
+  in `pytest-order` 
+ 
 ## [Version 0.8.1](https://pypi.org/project/pytest-order/0.8.1/) (2020-11-02)
 
 ### Added

docs/source/index.rst

@@ -275,7 +275,8 @@ question is ordered behind all other tests.
   The `pytest-dependency <https://pypi.org/project/pytest-dependency/>`__
   plugin also manages dependencies between tests (skips tests that depend
   on skipped or failed tests), but doesn't do any ordering. You can combine
-  both plugins if you need both options.
+  both plugins if you need both options--see :ref:`order-dependencies`
+  below for more information.
 
 Configuration
 =============
@@ -379,6 +380,55 @@ then from the end if there are negative numbers, and the rest will be in
 between (e.g. between positive and negative numbers), as it is without this
 option.
 
+.. _order-dependencies:
+
+``--order-dependencies``
+------------------------
+This defines the behavior if the ``pytest-dependency`` plugin is used.
+By default, ``dependency`` marks are only considered if they coexist with an
+``order`` mark. In this case it is checked if the ordering would break the
+dependency, and is ignored if this is the case. Consider the following:
+
+.. code:: python
+
+ import pytest
+
+ def test_a():
+     assert True
+
+ @pytest.mark.dependency(depends=['test_a'])
+ @pytest.mark.order("first")
+ def test_b():
+     assert True
+
+In this case, the ordering would break the dependency and is therefore
+ignored. This behavior is independent of the option. Now consider the
+following tests:
+
+.. code:: python
+
+ import pytest
+
+ @pytest.mark.dependency(depends=['test_b'])
+ def test_a():
+     assert True
+
+ def test_b():
+     assert True
+
+By default, ``test_a`` is not run, because it depends on ``test_b``, which
+is only run after ``test_b``. If you use ``--order-dependencies``, this will
+change--the tests will now be reordered according to the dependency and both
+run. Note that a similar feature may be added to ``pytest-dependency`` -
+if this is done, this option will not be needed, but for the time being you
+can use both plugins together to get this behavior.
+Note that ``pytest-order`` does not replace ``pytest-dependency``--it just
+adds ordering to the existing functionality if needed.
+
+.. note::
+  This feature is considered experimental. It will not handle all cases of
+  defined dependencies. If there is sufficient demand (reflected in issues),
+  this may be expanded.
 
 Miscellaneous
 =============

old_issues.md

@@ -37,8 +37,8 @@ Tracks the state of all open issues in pytest-ordering for reference.
 - [Unknown mark warning](https://github.com/ftobia/pytest-ordering/issues/57)  
   obsolete with registered marker :heavy_check_mark:
 - [pytest-ordering doesn't honor test dependencies](https://github.com/ftobia/pytest-ordering/issues/58)  
-  unclear what we can do here - maybe handle dependency markers like 
-  before/after markers? :thought_balloon: 
+  - ignore ordering if it would break a dependency
+  - added configuration option for ordering all dependencies :heavy_check_mark:
 - [should pytest-ordering be deprecated in favor of pytest-dependency?](https://github.com/ftobia/pytest-ordering/issues/59)  
   has been answered (`pytest-dependency` does not support ordering) :heavy_check_mark: 
 - [py.test ordering doesn't works when methods with order greater than 9 are present](https://github.com/ftobia/pytest-ordering/issues/61)  

pytest_order/__init__.py

@@ -77,6 +77,10 @@ def pytest_addoption(parser):
                     dest="sparse_ordering",
                     help="If there are gaps between ordinals they are filled "
                          "with unordered tests.")
+    group.addoption("--order-dependencies", action="store_true",
+                    dest="order_dependencies",
+                    help="If set, dependencies added by pytest-dependency will"
+                         "be ordered if needed.")
 
 
 class OrderingPlugin(object):
@@ -88,14 +92,50 @@ class OrderingPlugin(object):
     """
 
 
+class Settings:
+    sparse_ordering = False
+    order_dependencies = False
+    scope = "session"
+
+    @classmethod
+    def initialize(cls, config):
+        cls.sparse_ordering = config.getoption("sparse_ordering")
+        cls.order_dependencies = config.getoption("order_dependencies")
+        scope = config.getoption("order_scope")
+        if scope in ("session", "module", "class"):
+            cls.scope = scope
+        else:
+            if scope is not None:
+                warn("Unknown order scope '{}', ignoring it. "
+                     "Valid scopes are 'session', 'module' and 'class'."
+                     .format(scope))
+            cls.scope = "session"
+
+
 def get_filename(item):
     name = item.location[0]
     if os.sep in name:
         name = item.location[0].rsplit(os.sep, 1)[1]
     return name[:-3]
 
 
-def mark_binning(item, keys, start, end, before, after, unordered):
+def mark_binning(item, keys, start, end, before, after, unordered, alias):
+    if ("dependency" in keys and
+            (Settings.order_dependencies or "order" in keys)):
+        # always order dependencies if an order mark is present
+        # otherwise only if order-dependencies is set
+        mark = item.get_closest_marker("dependency")
+        prefix = get_filename(item) + "."
+        dependent_mark = mark.kwargs.get("depends")
+        if dependent_mark:
+            for name in dependent_mark:
+                if "." not in name:
+                    name = prefix + name
+                after.setdefault(name, []).append(item)
+        name_mark = mark.kwargs.get("name")
+        if name_mark:
+            alias[prefix + name_mark] = prefix + item.name
+
     if "order" in keys:
         mark = item.get_closest_marker("order")
         order = mark.args[0] if mark.args else None
@@ -156,22 +196,22 @@ def insert_after(name, items, sort):
     return False
 
 
-def do_modify_items(items, sparse_ordering):
+def do_modify_items(items):
     before_item = {}
     after_item = {}
     start_item = {}
     end_item = {}
     unordered_list = []
+    alias_names = {}
 
     for item in items:
         mark_binning(item, item.keywords.keys(), start_item, end_item,
-                     before_item, after_item, unordered_list)
+                     before_item, after_item, unordered_list, alias_names)
 
     start_item = sorted(start_item.items())
     end_item = sorted(end_item.items())
 
-    sorted_list = sort_numbered_items(start_item, end_item, unordered_list,
-                                      sparse_ordering)
+    sorted_list = sort_numbered_items(start_item, end_item, unordered_list)
 
     still_left = 0
     length = len(before_item) + len(after_item)
@@ -187,7 +227,8 @@ def do_modify_items(items, sparse_ordering):
 
         remove_labels = []
         for label, after in after_item.items():
-            if insert_after(label, after, sorted_list):
+            name = alias_names[label] if label in alias_names else label
+            if insert_after(name, after, sorted_list):
                 remove_labels.append(label)
         for label in remove_labels:
             del after_item[label]
@@ -207,11 +248,11 @@ def do_modify_items(items, sparse_ordering):
     return sorted_list
 
 
-def sort_numbered_items(start_item, end_item, unordered_list, sparse_ordering):
+def sort_numbered_items(start_item, end_item, unordered_list):
     sorted_list = []
     index = 0
     for entries in start_item:
-        if sparse_ordering:
+        if Settings.sparse_ordering:
             while entries[0] > index and unordered_list:
                 sorted_list.append(unordered_list.pop(0))
                 index += 1
@@ -220,38 +261,28 @@ def sort_numbered_items(start_item, end_item, unordered_list, sparse_ordering):
     mid_index = len(sorted_list)
     index = -1
     for entries in reversed(end_item):
-        if sparse_ordering:
+        if Settings.sparse_ordering:
             while entries[0] < index and unordered_list:
                 sorted_list.insert(mid_index, unordered_list.pop())
                 index -= 1
-        for item in reversed(entries[1]):
-            sorted_list.insert(mid_index, item)
-        index += len(entries[1])
-    for unordered_item in reversed(unordered_list):
-        sorted_list.insert(mid_index, unordered_item)
+        sorted_list[mid_index:mid_index] = entries[1]
+        index -= len(entries[1])
+    sorted_list[mid_index:mid_index] = unordered_list
     return sorted_list
 
 
 def modify_items(session, config, items):
-    sparse_ordering = config.getoption("sparse_ordering")
-    scope = config.getoption("order_scope")
-    if scope not in ("session", "module", "class"):
-        if scope is not None:
-            warn("Unknown order scope '{}', ignoring it. "
-                 "Valid scopes are 'session', 'module' and 'class'."
-                 .format(scope))
-        scope = "session"
-    if scope == "session":
-        sorted_list = do_modify_items(items, sparse_ordering)
-    elif scope == "module":
+    Settings.initialize(config)
+    if Settings.scope == "session":
+        sorted_list = do_modify_items(items)
+    elif Settings.scope == "module":
         module_items = OrderedDict()
         for item in items:
             module_path = item.nodeid[:item.nodeid.index("::")]
             module_items.setdefault(module_path, []).append(item)
         sorted_list = []
         for module_item_list in module_items.values():
-            sorted_list.extend(do_modify_items(
-                module_item_list, sparse_ordering))
+            sorted_list.extend(do_modify_items(module_item_list))
     else:  # class scope
         class_items = OrderedDict()
         for item in items:
@@ -262,7 +293,6 @@ def modify_items(session, config, items):
             class_items.setdefault(class_path, []).append(item)
         sorted_list = []
         for class_item_list in class_items.values():
-            sorted_list.extend(do_modify_items(
-                class_item_list, sparse_ordering))
+            sorted_list.extend(do_modify_items(class_item_list))
 
     items[:] = sorted_list

tests/test_dependency.py

@@ -0,0 +1,129 @@
+# -*- coding: utf-8 -*-
+
+import pytest
+
+import pytest_order
+from utils import write_test, assert_test_order
+
+
+def test_ignore_order_with_dependency(test_path, capsys):
+    tests_content = """
+import pytest
+
+def test_a():
+    pass
+
+@pytest.mark.dependency(depends=['test_a'])
+@pytest.mark.order("first")
+def test_b():
+    pass
+    """
+    write_test(test_path, tests_content)
+    pytest.main(["-v", test_path], [pytest_order])
+    out, err = capsys.readouterr()
+    assert_test_order(["test_a", "test_b"], out)
+
+
+def test_order_with_dependency(test_path, capsys):
+    tests_content = """
+import pytest
+
+@pytest.mark.dependency(depends=['test_b'])
+@pytest.mark.order("second")
+def test_a():
+    pass
+
+def test_b():
+    pass
+    """
+    write_test(test_path, tests_content)
+    pytest.main(["-v", test_path], [pytest_order])
+    out, err = capsys.readouterr()
+    assert_test_order(["test_b", "test_a"], out)
+
+
+def test_dependency_already_ordered(test_path, capsys):
+    tests_content = """
+import pytest
+
+def test_a():
+    pass
+
+@pytest.mark.dependency(depends=['test_a'])
+def test_b():
+    pass
+    """
+    write_test(test_path, tests_content)
+    pytest.main(["-v", test_path], [pytest_order])
+    out, err = capsys.readouterr()
+    assert_test_order(["test_a", "test_b"], out)
+    pytest.main(["-v", "--order-dependencies", test_path], [pytest_order])
+    out, err = capsys.readouterr()
+    assert_test_order(["test_a", "test_b"], out)
+
+
+def test_order_dependency(test_path, capsys):
+    tests_content = """
+import pytest
+
+@pytest.mark.dependency(depends=['test_b'])
+def test_a():
+    pass
+
+def test_b():
+    pass
+    """
+    write_test(test_path, tests_content)
+    pytest.main(["-v", test_path], [pytest_order])
+    out, err = capsys.readouterr()
+    assert_test_order(["test_a", "test_b"], out)
+    pytest.main(["-v", "--order-dependencies", test_path], [pytest_order])
+    out, err = capsys.readouterr()
+    assert_test_order(["test_b", "test_a"], out)
+
+
+def test_order_multiple_dependencies(test_path, capsys):
+    tests_content = """
+import pytest
+
+@pytest.mark.dependency(depends=["test_b", "test_c"])
+def test_a():
+    pass
+
+def test_b():
+    pass
+
+def test_c():
+    pass
+    """
+    write_test(test_path, tests_content)
+    pytest.main(["-v", test_path], [pytest_order])
+    out, err = capsys.readouterr()
+    assert_test_order(["test_a", "test_b", "test_c"], out)
+    pytest.main(["-v", "--order-dependencies", test_path], [pytest_order])
+    out, err = capsys.readouterr()
+    assert_test_order(["test_b", "test_c", "test_a"], out)
+
+
+def test_order_named_dependency(test_path, capsys):
+    tests_content = """
+import pytest
+
+@pytest.mark.dependency(depends=["my_test"])
+def test_a():
+    pass
+
+@pytest.mark.dependency(name="my_test")
+def test_b():
+    pass
+
+def test_c():
+    pass
+    """
+    write_test(test_path, tests_content)
+    pytest.main(["-v", test_path], [pytest_order])
+    out, err = capsys.readouterr()
+    assert_test_order(["test_a", "test_b", "test_c"], out)
+    pytest.main(["-v", "--order-dependencies", test_path], [pytest_order])
+    out, err = capsys.readouterr()
+    assert_test_order(["test_b", "test_a", "test_c"], out)