diff --git a/doc/develop/test/index.rst b/doc/develop/test/index.rst index 9f43f86f8bc86..c5f96b400caee 100644 --- a/doc/develop/test/index.rst +++ b/doc/develop/test/index.rst @@ -12,4 +12,3 @@ Testing pytest coverage BabbleSim - ztest_deprecated diff --git a/doc/develop/test/ztest.rst b/doc/develop/test/ztest.rst index 8e5ccddd6a92c..6db3867a09c67 100644 --- a/doc/develop/test/ztest.rst +++ b/doc/develop/test/ztest.rst @@ -263,7 +263,7 @@ prj.conf :language: text :linenos: -src/main.c (see :ref:`best practices `) +src/main.c .. literalinclude:: ../../../samples/subsys/testsuite/integration/src/main.c :language: c diff --git a/doc/develop/test/ztest_deprecated.rst b/doc/develop/test/ztest_deprecated.rst deleted file mode 100644 index 2bd7194c55b8f..0000000000000 --- a/doc/develop/test/ztest_deprecated.rst +++ /dev/null @@ -1,266 +0,0 @@ -.. _test-framework-deprecated: - -ZTest Deprecated APIs -##################### - -Ztest is currently being migrated to a new API, this documentation provides information about the -deprecated APIs which will eventually be removed. See :ref:`Test Framework ` for the new API. -Similarly, ZTest's mocking framework is also deprecated (see :ref:`Mocking via FFF `). - -Quick start - Unit testing -************************** - -Ztest can be used for unit testing. This means that rather than including the -entire Zephyr OS for testing a single function, you can focus the testing -efforts into the specific module in question. This will speed up testing since -only the module will have to be compiled in, and the tested functions will be -called directly. - -Since you won't be including basic kernel data structures that most code -depends on, you have to provide function stubs in the test. Ztest provides -some helpers for mocking functions, as demonstrated below. - -In a unit test, mock objects can simulate the behavior of complex real objects -and are used to decide whether a test failed or passed by verifying whether an -interaction with an object occurred, and if required, to assert the order of -that interaction. - -.. _main_c_bp: - -Best practices for declaring the test suite -=========================================== - -*twister* and other validation tools need to obtain the list of -subcases that a Zephyr *ztest* test image will expose. - -.. admonition:: Rationale - - This all is for the purpose of traceability. It's not enough to - have only a semaphore test project. We also need to show that we - have testpoints for all APIs and functionality, and we trace back - to documentation of the API, and functional requirements. - - The idea is that test reports show results for every sub-testcase - as passed, failed, blocked, or skipped. Reporting on only the - high-level test project level, particularly when tests do too - many things, is too vague. - -There exist two alternatives to writing tests. The first, and more verbose, -approach is to directly declare and run the test suites. -Here is a generic template for a test showing the expected use of -:c:func:`ztest_test_suite`: - -.. code-block:: C - - #include - - extern void test_sometest1(void); - extern void test_sometest2(void); - #ifndef CONFIG_WHATEVER /* Conditionally skip test_sometest3 */ - void test_sometest3(void) - { - ztest_test_skip(); - } - #else - extern void test_sometest3(void); - #endif - extern void test_sometest4(void); - ... - - void test_main(void) - { - ztest_test_suite(common, - ztest_unit_test(test_sometest1), - ztest_unit_test(test_sometest2), - ztest_unit_test(test_sometest3), - ztest_unit_test(test_sometest4) - ); - ztest_run_test_suite(common); - } - -Alternatively, it is possible to split tests across multiple files using -:c:func:`ztest_register_test_suite` which bypasses the need for ``extern``: - -.. code-block:: C - - #include - - void test_sometest1(void) { - zassert_true(1, "true"); - } - - ztest_register_test_suite(common, NULL, - ztest_unit_test(test_sometest1) - ); - -The above sample simple registers the test suite and uses a ``NULL`` pragma -function (more on that later). It is important to note that the test suite isn't -directly run in this file. Instead two alternatives exist for running the suite. -First, if to do nothing. A default ``test_main`` function is provided by -ztest. This is the preferred approach if the test doesn't involve a state and -doesn't require use of the pragma. - -In cases of an integration test it is possible that some general state needs to -be set between test suites. This can be thought of as a state diagram in which -``test_main`` simply goes through various actions that modify the board's -state and different test suites need to run. This is achieved in the following: - -.. code-block:: C - - #include - - struct state { - bool is_hibernating; - bool is_usb_connected; - } - - static bool pragma_always(const void *state) - { - return true; - } - - static bool pragma_not_hibernating_not_connected(const void *s) - { - struct state *state = s; - return !state->is_hibernating && !state->is_usb_connected; - } - - static bool pragma_usb_connected(const void *s) - { - return ((struct state *)s)->is_usb_connected; - } - - ztest_register_test_suite(baseline, pragma_always, - ztest_unit_test(test_case0)); - ztest_register_test_suite(before_usb, pragma_not_hibernating_not_connected, - ztest_unit_test(test_case1), - ztest_unit_test(test_case2)); - ztest_register_test_suite(with_usb, pragma_usb_connected,, - ztest_unit_test(test_case3), - ztest_unit_test(test_case4)); - - void test_main(void) - { - struct state state; - - /* Should run `baseline` test suite only. */ - ztest_run_registered_test_suites(&state); - - /* Simulate power on and update state. */ - emulate_power_on(); - /* Should run `baseline` and `before_usb` test suites. */ - ztest_run_registered_test_suites(&state); - - /* Simulate plugging in a USB device. */ - emulate_plugging_in_usb(); - /* Should run `baseline` and `with_usb` test suites. */ - ztest_run_registered_test_suites(&state); - - /* Verify that all the registered test suites actually ran. */ - ztest_verify_all_registered_test_suites_ran(); - } - -For *twister* to parse source files and create a list of subcases, -the declarations of :c:func:`ztest_test_suite` and -:c:func:`ztest_register_test_suite` must follow a few rules: - -- one declaration per line - -- conditional execution by using :c:func:`ztest_test_skip` - -What to avoid: - -- packing multiple testcases in one source file - - .. code-block:: C - - void test_main(void) - { - #ifdef TEST_feature1 - ztest_test_suite(feature1, - ztest_unit_test(test_1a), - ztest_unit_test(test_1b), - ztest_unit_test(test_1c) - ); - ztest_run_test_suite(feature1); - #endif - - #ifdef TEST_feature2 - ztest_test_suite(feature2, - ztest_unit_test(test_2a), - ztest_unit_test(test_2b) - ); - ztest_run_test_suite(feature2); - #endif - } - - -- Do not use ``#if`` - - .. code-block:: C - - ztest_test_suite(common, - ztest_unit_test(test_sometest1), - ztest_unit_test(test_sometest2), - #ifdef CONFIG_WHATEVER - ztest_unit_test(test_sometest3), - #endif - ztest_unit_test(test_sometest4), - ... - -- Do not add comments on lines with a call to :c:func:`ztest_unit_test`: - - .. code-block:: C - - ztest_test_suite(common, - ztest_unit_test(test_sometest1), - ztest_unit_test(test_sometest2) /* will fail */, - /* will fail! */ ztest_unit_test(test_sometest3), - ztest_unit_test(test_sometest4), - ... - -- Do not define multiple definitions of unit / user unit test case per - line - - - .. code-block:: C - - ztest_test_suite(common, - ztest_unit_test(test_sometest1), ztest_unit_test(test_sometest2), - ztest_unit_test(test_sometest3), - ztest_unit_test(test_sometest4), - ... - - -Other questions: - -- Why not pre-scan with CPP and then parse? or post scan the ELF file? - - If C pre-processing or building fails because of any issue, then we - won't be able to tell the subcases. - -- Why not declare them in the YAML testcase description? - - A separate testcase description file would be harder to maintain - than just keeping the information in the test source files - themselves -- only one file to update when changes are made - eliminates duplication. - -Mocking -******* - -These functions allow abstracting callbacks and related functions and -controlling them from specific tests. You can enable the mocking framework by -setting :kconfig:option:`CONFIG_ZTEST_MOCKING` to "y" in the configuration file of the -test. The amount of concurrent return values and expected parameters is -limited by :kconfig:option:`CONFIG_ZTEST_PARAMETER_COUNT`. - -Here is an example for configuring the function ``expect_two_parameters`` to -expect the values ``a=2`` and ``b=3``, and telling ``returns_int`` to return -``5``: - -.. literalinclude:: mocking.c - :language: c - :linenos: - -.. doxygengroup:: ztest_mock