Merge pull request #95 from smarie/fix_issue_92

New `lazy_value` for parametrize_plus

Author: smarie

docs/index.md

@@ -8,23 +8,21 @@
 
 !!! success "You can now use `pytest.param` in the argvalues provided to `fixture_union`, `param_fixture[s]` and `parametrize_plus`, just as you do in `pytest`. See [pytest documentation](https://docs.pytest.org/en/stable/example/parametrize.html#set-marks-or-test-id-for-individual-parametrized-test)"
 
+!!! success "New `lazy_value` feature for parametrize, [check it out](#parametrize_plus) !"
+
 !!! warning "Test execution order"
     Installing pytest-cases now has effects on the order of `pytest` tests execution, even if you do not use its features. One positive side effect is that it fixed [pytest#5054](https://github.com/pytest-dev/pytest/issues/5054). But if you see less desirable ordering please [report it](https://github.com/smarie/python-pytest-cases/issues).
 
-!!! warning "New aliases"
-    `pytest_fixture_plus` and `pytest_parametrize_plus` were renamed to `fixture_plus` and `parametrize_plus` in order for pytest (pluggy) not to think they were hooks. Old aliases will stay around for a few versions, with a deprecation warning. See [#71](https://github.com/smarie/python-pytest-cases/issues/71).
-
 Did you ever think that most of your test functions were actually *the same test code*, but with *different data inputs* and expected results/exceptions ?
 
 `pytest-cases` leverages `pytest` and its great `@pytest.mark.parametrize` decorator, so that you can **separate your test cases from your test functions**. For example with `pytest-cases` you can now write your tests with the following pattern:
 
  * on one hand, the usual `test_xxxx.py` file containing your test functions
  * on the other hand, a new `test_xxxx_cases.py` containing your cases functions
 
-`pytest-cases` is fully compliant with [pytest-steps](https://smarie.github.io/python-pytest-steps/) so you can create test suites with several steps and send each case on the full suite. See [usage page for details](./usage/advanced/#test-suites-several-steps-on-each-case).
-
 In addition, `pytest-cases` improves `pytest`'s fixture mechanism to support "fixture unions". This is a **major change** in the internal `pytest` engine, unlocking many possibilities such as using fixture references as parameter values in a test function. See [below](#fixture_union).
 
+`pytest-cases` is fully compliant with [pytest-steps](https://smarie.github.io/python-pytest-steps/) so you can create test suites with several steps and send each case on the full suite. See [usage page for details](./usage/advanced/#test-suites-several-steps-on-each-case).
 
 ## Installing
 
@@ -363,27 +361,35 @@ Fixture unions are a **major change** in the internal pytest engine, as fixture
 
 ### `@parametrize_plus`
 
-`@parametrize_plus` is a replacement for `@pytest.mark.parametrize` that allows you to include references to fixtures in the parameter values. Simply use `fixture_ref(<fixture>)` in the parameter values, where `<fixture>` can be the fixture name or fixture function.
+`@parametrize_plus` is a replacement for `@pytest.mark.parametrize` that allows you to include references to fixtures and to value-generating functions in the parameter values. 
 
-For example:
+ - Simply use `fixture_ref(<fixture>)` in the parameter values, where `<fixture>` can be the fixture name or fixture function.
+ - if you do not wish to create a fixture, you can also use `lazy_value(<function>)`
+ - Note that when parametrizing several argnames, both `fixture_ref` and `lazy_value` can be used *as* the tuple, or *in* the tuple. Several `fixture_ref` and/or `lazy_value` can be used in the same tuple, too. 
+
+For example, with a single argument:
 
 ```python
 import pytest
-from pytest_cases import parametrize_plus, fixture_plus, fixture_ref
+from pytest_cases import parametrize_plus, fixture_plus, fixture_ref, lazy_value
 
 @pytest.fixture
 def world_str():
     return 'world'
 
+def whatfun():
+    return 'what'
+
 @fixture_plus
 @parametrize_plus('who', [fixture_ref(world_str), 
-                                 'you'])
+                          'you'])
 def greetings(who):
     return 'hello ' + who
 
 @parametrize_plus('main_msg', ['nothing', 
-                                      fixture_ref(world_str), 
-                                      fixture_ref(greetings)])
+                               fixture_ref(world_str),
+                               lazy_value(whatfun), 
+                               fixture_ref(greetings)])
 @pytest.mark.parametrize('ending', ['?', '!'])
 def test_prints(main_msg, ending):
     print(main_msg + ending)
@@ -393,22 +399,24 @@ yields the following
 
 ```bash
 > pytest -s -v
-collected 9 items
-test_prints[test_prints_main_msg_is_0-nothing-?] nothing?                                 PASSED
-test_prints[test_prints_main_msg_is_0-nothing-!] nothing!                                 PASSED
-test_prints[test_prints_main_msg_is_world_str-?] world?                                   PASSED
-test_prints[test_prints_main_msg_is_world_str-!] world!                                   PASSED
-test_prints[test_prints_main_msg_is_greetings-greetings_who_is_world_str-?] hello world?  PASSED
-test_prints[test_prints_main_msg_is_greetings-greetings_who_is_world_str-!] hello world!  PASSED
-test_prints[test_prints_main_msg_is_greetings-greetings_who_is_1-you-?] hello you?        PASSED
-test_prints[test_prints_main_msg_is_greetings-greetings_who_is_1-you-!] hello you!        PASSED
+collected 10 items
+test_prints[main_msg_is_nothing-?] PASSED       [ 10%]nothing?
+test_prints[main_msg_is_nothing-!] PASSED       [ 20%]nothing!
+test_prints[main_msg_is_world_str-?] PASSED     [ 30%]world?
+test_prints[main_msg_is_world_str-!] PASSED     [ 40%]world!
+test_prints[main_msg_is_whatfun-?] PASSED       [ 50%]what?
+test_prints[main_msg_is_whatfun-!] PASSED       [ 60%]what!
+test_prints[main_msg_is_greetings-who_is_world_str-?] PASSED [ 70%]hello world?
+test_prints[main_msg_is_greetings-who_is_world_str-!] PASSED [ 80%]hello world!
+test_prints[main_msg_is_greetings-who_is_you-?] PASSED [ 90%]hello you?
+test_prints[main_msg_is_greetings-who_is_you-!] PASSED [100%]hello you!
 ```
 
-As you can see, the ids are a bit more explicit than usual. As opposed to `fixture_union`, the style of these ids is not configurable for now but feel free to propose alternatives in the [issues page](https://github.com/smarie/python-pytest-cases/issues). 
-
 You can also mark any of the argvalues with `pytest.mark` to pass a custom id or a custom "skip" or "fail" mark, just as you do in `pytest`. See [pytest documentation](https://docs.pytest.org/en/stable/example/parametrize.html#set-marks-or-test-id-for-individual-parametrized-test).
 
-Note: for this to be performed, the parameters are replaced with a union fixture. Therefore the relative priority order of these parameters with other standard `pytest.mark.parametrize` parameters that you would place on the same function, will get impacted. You may solve this by replacing your other `@pytest.mark.parametrize` calls with `param_fixture`s (see [above](#param_fixtures).)
+As you can see in the example above, the default ids are a bit more explicit than usual when you use at least one `fixture_ref`. This is because the parameters need to be replaced with a fixture union that will "switch" between alternative groups of parameters, and the appropriate fixtures referenced. As opposed to `fixture_union`, the style of these ids is not configurable for now, but feel free to propose alternatives in the [issues page](https://github.com/smarie/python-pytest-cases/issues). Note that this does not happen if you only use `lazy_value`s, as they do not require to create a fixture union behind the scenes.
+
+Another consequence of using `fixture_ref` is that the priority order of the parameters, relative to other standard `pytest.mark.parametrize` parameters that you would place on the same function, will get impacted. You may solve this by replacing your other `@pytest.mark.parametrize` calls with `param_fixture`s so that all the parameters are fixtures (see [above](#param_fixtures).)
 
 ### passing a `hook`
 

pytest_cases/__init__.py

@@ -1,6 +1,6 @@
 from .fixture_core1_unions import fixture_union, NOT_USED, unpack_fixture, ignore_unused
 from .fixture_core2 import pytest_fixture_plus, fixture_plus, param_fixtures, param_fixture
-from .fixture_parametrize_plus import pytest_parametrize_plus, parametrize_plus, fixture_ref
+from .fixture_parametrize_plus import pytest_parametrize_plus, parametrize_plus, fixture_ref, lazy_value
 
 from .case_funcs import case_name, test_target, case_tags, cases_generator
 from .case_parametrizer import cases_data, CaseDataGetter, unfold_expected_err, get_all_cases, THIS_MODULE, \
@@ -24,6 +24,7 @@
     'case_funcs', 'case_parametrizer', 'fixture_core1_unions', 'fixture_core2', 'fixture_parametrize_plus',
     # all symbols imported above
     # --cases_funcs
+    'lazy_value',
     'case_name',  'test_target', 'case_tags', 'cases_generator',
     # --main_fixtures
     'cases_fixture', 'pytest_fixture_plus', 'fixture_plus', 'param_fixtures', 'param_fixture', 'ignore_unused',

pytest_cases/common_pytest.py

@@ -467,6 +467,17 @@ def get_marked_parameter_id(v):
 except ImportError:  # pytest 2.x
     from _pytest.mark import MarkDecorator
 
+    def ParameterSet(values, id, marks):
+        """ Dummy function (not a class) used only by parametrize_plus """
+        if id is not None:
+            raise ValueError("This should not happen as `pytest.param` does not exist in pytest 2")
+        for m in marks:
+            values = pytest.mark()
+            raise ValueError("TODO")
+
+        # smart unpack is required for compatibility
+        return values[0] if len(values) == 1 else values
+
     def is_marked_parameter_value(v):
         return isinstance(v, MarkDecorator)
 

pytest_cases/fixture_core1_unions.py

@@ -122,6 +122,9 @@ def is_fixture_union_params(params):
         if len(params) < 1:
             return False
         else:
+            if getattr(params, '__module__', '').startswith('pytest_cases'):
+                # a value_ref_tuple or another proxy object created somewhere in our code, not a list
+                return False
             p0 = params[0]
             if is_marked_parameter_value(p0):
                 p0 = get_marked_parameter_values(p0)[0]

pytest_cases/fixture_core2.py

@@ -33,6 +33,7 @@ def param_fixture(argname,           # type: str
                   ids=None,          # type: Union[Callable, List[str]]
                   scope="function",  # type: str
                   hook=None,         # type: Callable[[Callable], Callable]
+                  debug=False,       # type: bool
                   **kwargs):
     """
     Identical to `param_fixtures` but for a single parameter name, so that you can assign its output to a single
@@ -62,6 +63,7 @@ def test_uses_param(my_parameter, fixture_uses_param):
         will be called everytime a fixture is about to be created. It will receive a single argument (the function
         implementing the fixture) and should return the function to use. For example you can use `saved_fixture` from
         `pytest-harvest` as a hook in order to save all such created fixtures in the fixture store.
+    :param debug: print debug messages on stdout to analyze fixture creation (use pytest -s to see them)
     :param kwargs: any other argument for 'fixture'
     :return: the create fixture
     """
@@ -74,7 +76,7 @@ def test_uses_param(my_parameter, fixture_uses_param):
     caller_module = get_caller_module()
 
     return _create_param_fixture(caller_module, argname, argvalues, autouse=autouse, ids=ids, scope=scope,
-                                 hook=hook, **kwargs)
+                                 hook=hook, debug=debug, **kwargs)
 
 
 def _create_param_fixture(caller_module,
@@ -85,6 +87,7 @@ def _create_param_fixture(caller_module,
                           scope="function",  # type: str
                           hook=None,         # type: Callable[[Callable], Callable]
                           auto_simplify=False,
+                          debug=False,
                           **kwargs):
     """ Internal method shared with param_fixture and param_fixtures """
 
@@ -99,13 +102,19 @@ def _create_param_fixture(caller_module,
         def __param_fixture():
             return argvalue_to_return
 
+        if debug:
+            print("Creating unparametrized fixture %r returning %r" % (argname, argvalue_to_return))
+
         fix = fixture_plus(name=argname, scope=scope, autouse=autouse, ids=ids, hook=hook, **kwargs)(__param_fixture)
     else:
         # create the fixture - set its name so that the optional hook can read it easily
         @with_signature("%s(request)" % argname)
         def __param_fixture(request):
             return request.param
 
+        if debug:
+            print("Creating parametrized fixture %r returning %r" % (argname, argvalues))
+
         fix = fixture_plus(name=argname, scope=scope, autouse=autouse, params=argvalues, ids=ids,
                            hook=hook, **kwargs)(__param_fixture)
 
@@ -122,6 +131,7 @@ def param_fixtures(argnames,
                    ids=None,          # type: Union[Callable, List[str]]
                    scope="function",  # type: str
                    hook=None,         # type: Callable[[Callable], Callable]
+                   debug=False,       # type: bool
                    **kwargs):
     """
     Creates one or several "parameters" fixtures - depending on the number or coma-separated names in `argnames`. The
@@ -156,17 +166,33 @@ def test_uses_param2(arg1, arg2, fixture_uses_param2):
         will be called everytime a fixture is about to be created. It will receive a single argument (the function
         implementing the fixture) and should return the function to use. For example you can use `saved_fixture` from
         `pytest-harvest` as a hook in order to save all such created fixtures in the fixture store.
+    :param debug: print debug messages on stdout to analyze fixture creation (use pytest -s to see them)
     :param kwargs: any other argument for the created 'fixtures'
     :return: the created fixtures
     """
-    created_fixtures = []
     argnames_lst = get_param_argnames_as_list(argnames)
 
     caller_module = get_caller_module()
 
     if len(argnames_lst) < 2:
         return _create_param_fixture(caller_module, argnames, argvalues, autouse=autouse, ids=ids, scope=scope,
-                                     hook=hook, **kwargs)
+                                     hook=hook, debug=debug, **kwargs)
+    else:
+        return _create_params_fixture(caller_module, argnames_lst, argvalues, autouse=autouse, ids=ids, scope=scope,
+                                      hook=hook, debug=debug, **kwargs)
+
+
+def _create_params_fixture(caller_module,
+                           argnames_lst,      # type: Sequence[str]
+                           argvalues,         # type: Sequence[Any]
+                           autouse=False,     # type: bool
+                           ids=None,          # type: Union[Callable, List[str]]
+                           scope="function",  # type: str
+                           hook=None,         # type: Callable[[Callable], Callable]
+                           debug=False,       # type: bool
+                           **kwargs):
+    argnames = ','.join(argnames_lst)
+    created_fixtures = []
 
     # create the root fixture that will contain all parameter values
     # note: we sort the list so that the first in alphabetical order appears first. Indeed pytest uses this order.
@@ -176,6 +202,9 @@ def test_uses_param2(arg1, arg2, fixture_uses_param2):
     root_fixture_name = check_name_available(caller_module, root_fixture_name, if_name_exists=CHANGE,
                                              caller=param_fixtures)
 
+    if debug:
+        print("Creating parametrized 'root' fixture %r returning %r" % (root_fixture_name, argvalues))
+
     @fixture_plus(name=root_fixture_name, autouse=autouse, scope=scope, hook=hook, **kwargs)
     @pytest.mark.parametrize(argnames, argvalues, ids=ids)
     @with_signature("%s(%s)" % (root_fixture_name, argnames))
@@ -191,11 +220,16 @@ def _root_fixture(**_kwargs):
         # To fix late binding issue with `param_idx` we add an extra layer of scope: a factory function
         # See https://stackoverflow.com/questions/3431676/creating-functions-in-a-loop
         def _create_fixture(_param_idx):
+
+            if debug:
+                print("Creating nonparametrized 'view' fixture %r returning %r[%s]" % (argname, root_fixture_name, _param_idx))
+
             @fixture_plus(name=argname, scope=scope, autouse=autouse, hook=hook, **kwargs)
             @with_signature("%s(%s)" % (argname, root_fixture_name))
             def _param_fixture(**_kwargs):
                 params = _kwargs.pop(root_fixture_name)
                 return params[_param_idx]
+
             return _param_fixture
 
         # create it
@@ -406,6 +440,8 @@ def _decorate_fixture_plus(fixture_func,
 
     # --common routine used below. Fills kwargs with the appropriate names and values from fixture_params
     def _map_arguments(*_args, **_kwargs):
+        # todo better...
+        from .fixture_parametrize_plus import handle_lazy_args
         request = _kwargs['request'] if func_needs_request else _kwargs.pop('request')
 
         # populate the parameters
@@ -416,12 +452,12 @@ def _map_arguments(*_args, **_kwargs):
         for p_names, fixture_param_value in zip(params_names_or_name_combinations, _params):
             if len(p_names) == 1:
                 # a single parameter for that generated fixture (@pytest.mark.parametrize with a single name)
-                _kwargs[p_names[0]] = fixture_param_value
+                _kwargs[p_names[0]] = handle_lazy_args(fixture_param_value)
             else:
                 # several parameters for that generated fixture (@pytest.mark.parametrize with several names)
                 # unpack all of them and inject them in the kwargs
                 for old_p_name, old_p_value in zip(p_names, fixture_param_value):
-                    _kwargs[old_p_name] = old_p_value
+                    _kwargs[old_p_name] = handle_lazy_args(old_p_value)
 
         return _args, _kwargs