Skip to content

Commit 42a3b95

Browse files
committed
doc: twister: Update test naming and application diagram
Update Test Application diagram as well as Test Scenario and Test Case naming conventions to make them more clear and aligned to Ztest suite name included as a part of Test Case name. Signed-off-by: Dmitrii Golovanov <[email protected]>
1 parent 2b84965 commit 42a3b95

File tree

2 files changed

+34
-33
lines changed

2 files changed

+34
-33
lines changed

doc/develop/test/figures/twister_test_project.svg

Lines changed: 1 addition & 1 deletion
Loading

doc/develop/test/twister.rst

Lines changed: 33 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -219,57 +219,60 @@ Tests
219219

220220
Tests are detected by the presence of a ``testcase.yaml`` or a ``sample.yaml``
221221
files in the application's project directory. This test application
222-
configuration file may contain one or more entries in the tests section each
223-
identifying a test scenario.
222+
configuration file may contain one or more entries in the ``tests:`` section each
223+
identifying a Test Scenario.
224224

225225
.. _twister_test_project_diagram:
226226

227227
.. figure:: figures/twister_test_project.svg
228-
:alt: Twister and a Test applications' project.
228+
:alt: Twister and a Test application project.
229229
:figclass: align-center
230230

231-
Twister and a Test applications' project.
231+
Twister and a Test application project.
232232

233233

234234
Test application configurations are written using the YAML syntax and share the
235235
same structure as samples.
236236

237-
A test scenario is a set of conditions or variables, defined in test scenario
238-
entry, under which a set of test suites will be executed. Can be used
239-
interchangeably with test scenario entry.
237+
A Test Scenario is a set of conditions and variables defined in a Test Scenario
238+
entry, under which a set of Test Suites will be built and executed.
240239

241-
A test suite is a collection of test cases that are intended to be used to test
242-
a software program to ensure it meets certain requirements. The test cases in a
243-
test suite are often related or meant to be executed together.
240+
A Test Suite is a collection of Test Cases which are intended to be used to test
241+
a software program to ensure it meets certain requirements. The Test Cases in a
242+
Test Suite are either related or meant to be executed together.
244243

245-
The name of each test scenario needs to be unique in the context of the overall
244+
The name of each Test Scenario needs to be unique in the context of the overall
246245
test application and has to follow basic rules:
247246

248-
#. The format of the test scenario identifier shall be a string without any spaces or
247+
#. The format of the Test Scenario identifier shall be a string without any spaces or
249248
special characters (allowed characters: alphanumeric and [\_=]) consisting
250-
of multiple sections delimited with a dot (.).
249+
of multiple sections delimited with a dot (``.``).
251250

252-
#. Each test scenario identifier shall start with a section followed by a
253-
subsection separated by a dot. For example, a test scenario that covers
254-
semaphores in the kernel shall start with ``kernel.semaphore``.
251+
#. Each Test Scenario identifier shall start with a section name followed by a
252+
subsection names delimited with a dot (``.``). For example, a test scenario
253+
that covers semaphores in the kernel shall start with ``kernel.semaphore``.
255254

256-
#. All test scenario identifiers within a ``testcase.yaml`` file need to be unique. For
257-
example a ``testcase.yaml`` file covering semaphores in the kernel can have:
255+
#. All Test Scenario identifiers within a ``testcase.yaml`` file need to be unique.
256+
For example a ``testcase.yaml`` file covering semaphores in the kernel can have:
258257

259258
* ``kernel.semaphore``: For general semaphore tests
260259
* ``kernel.semaphore.stress``: Stress testing semaphores in the kernel.
261260

262-
#. Depending on the nature of the test, an identifier can consist of at least
263-
two sections:
261+
#. The full canonical name of a Test Suite is:
262+
``<Test Application Project path>/<Test Scenario identifier>``
264263

265-
* Ztest tests: The individual test cases in the ztest testsuite will be
266-
concatenated by dot (``.``) to the identifier in the ``testcase.yaml`` file
267-
generating unique identifiers for every test case in the suite.
264+
#. Depending on the Test Suite implementation, its Test Case identifiers consist
265+
of **at least three sections** delimited with a dot (``.``):
268266

269-
* Standalone tests and samples: This type of test should at least have 3
270-
sections concatnated by dot (``.``) in the test scenario identifier in the
271-
``testcase.yaml`` (or ``sample.yaml``) file.
272-
The last section of the name shall signify the test case itself.
267+
* **Ztest tests**:
268+
a Test Scenario identifier from the corresponding ``testcase.yaml`` file,
269+
a Ztest suite name, and a Ztest test name:
270+
``<Test Scenario identifier>.<Ztest suite name>.<Ztest test name>``
271+
272+
* **Standalone tests and samples**:
273+
a Test Scenario identifier from the corresponding ``testcase.yaml`` (or
274+
``sample.yaml``) file where the last section signifies the standalone
275+
Test Case name, for example: ``debug.coredump.logging_backend``.
273276

274277

275278
The following is an example test configuration with a few options that are
@@ -312,12 +315,10 @@ related to the sample and what is being demonstrated:
312315
tags: tests
313316
min_ram: 16
314317
315-
The full canonical name for each test scenario is:``<path to test application>/<test scenario identifier>``
316-
317-
A test scenario entry is a a block or entry starting with test scenario
318-
identifier in the YAML files.
318+
A Test Scenario entry in the ``tests:`` YAML dictionary has its Test Scenario
319+
identifier as a key.
319320

320-
Each test scenario entry in the test application configuration can define the
321+
Each Test Scenario entry in the Test Application configuration can define the
321322
following key/value pairs:
322323

323324
.. _test_config_args:

0 commit comments

Comments
 (0)