Skip to content

Commit 18451bc

Browse files
golowanownashif
authored andcommitted
doc: twister: Update test naming and application diagram
Update Test Application diagram as well as Test Scenario and Test Case naming conventions to make them more clear and aligned to Ztest suite name included as a part of Test Case name. Signed-off-by: Dmitrii Golovanov <[email protected]>
1 parent a72244f commit 18451bc

File tree

2 files changed

+34
-33
lines changed

2 files changed

+34
-33
lines changed

doc/develop/test/figures/twister_test_project.svg

Lines changed: 1 addition & 1 deletion
Loading

doc/develop/test/twister.rst

Lines changed: 33 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -223,57 +223,60 @@ Tests
223223

224224
Tests are detected by the presence of a ``testcase.yaml`` or a ``sample.yaml``
225225
files in the application's project directory. This test application
226-
configuration file may contain one or more entries in the tests section each
227-
identifying a test scenario.
226+
configuration file may contain one or more entries in the ``tests:`` section each
227+
identifying a Test Scenario.
228228

229229
.. _twister_test_project_diagram:
230230

231231
.. figure:: figures/twister_test_project.svg
232-
:alt: Twister and a Test applications' project.
232+
:alt: Twister and a Test application project.
233233
:figclass: align-center
234234

235-
Twister and a Test applications' project.
235+
Twister and a Test application project.
236236

237237

238238
Test application configurations are written using the YAML syntax and share the
239239
same structure as samples.
240240

241-
A test scenario is a set of conditions or variables, defined in test scenario
242-
entry, under which a set of test suites will be executed. Can be used
243-
interchangeably with test scenario entry.
241+
A Test Scenario is a set of conditions and variables defined in a Test Scenario
242+
entry, under which a set of Test Suites will be built and executed.
244243

245-
A test suite is a collection of test cases that are intended to be used to test
246-
a software program to ensure it meets certain requirements. The test cases in a
247-
test suite are often related or meant to be executed together.
244+
A Test Suite is a collection of Test Cases which are intended to be used to test
245+
a software program to ensure it meets certain requirements. The Test Cases in a
246+
Test Suite are either related or meant to be executed together.
248247

249-
The name of each test scenario needs to be unique in the context of the overall
248+
The name of each Test Scenario needs to be unique in the context of the overall
250249
test application and has to follow basic rules:
251250

252-
#. The format of the test scenario identifier shall be a string without any spaces or
251+
#. The format of the Test Scenario identifier shall be a string without any spaces or
253252
special characters (allowed characters: alphanumeric and [\_=]) consisting
254-
of multiple sections delimited with a dot (.).
253+
of multiple sections delimited with a dot (``.``).
255254

256-
#. Each test scenario identifier shall start with a section followed by a
257-
subsection separated by a dot. For example, a test scenario that covers
258-
semaphores in the kernel shall start with ``kernel.semaphore``.
255+
#. Each Test Scenario identifier shall start with a section name followed by a
256+
subsection names delimited with a dot (``.``). For example, a test scenario
257+
that covers semaphores in the kernel shall start with ``kernel.semaphore``.
259258

260-
#. All test scenario identifiers within a ``testcase.yaml`` file need to be unique. For
261-
example a ``testcase.yaml`` file covering semaphores in the kernel can have:
259+
#. All Test Scenario identifiers within a ``testcase.yaml`` file need to be unique.
260+
For example a ``testcase.yaml`` file covering semaphores in the kernel can have:
262261

263262
* ``kernel.semaphore``: For general semaphore tests
264263
* ``kernel.semaphore.stress``: Stress testing semaphores in the kernel.
265264

266-
#. Depending on the nature of the test, an identifier can consist of at least
267-
two sections:
265+
#. The full canonical name of a Test Suite is:
266+
``<Test Application Project path>/<Test Scenario identifier>``
268267

269-
* Ztest tests: The individual test cases in the ztest testsuite will be
270-
concatenated by dot (``.``) to the identifier in the ``testcase.yaml`` file
271-
generating unique identifiers for every test case in the suite.
268+
#. Depending on the Test Suite implementation, its Test Case identifiers consist
269+
of **at least three sections** delimited with a dot (``.``):
272270

273-
* Standalone tests and samples: This type of test should at least have 3
274-
sections concatnated by dot (``.``) in the test scenario identifier in the
275-
``testcase.yaml`` (or ``sample.yaml``) file.
276-
The last section of the name shall signify the test case itself.
271+
* **Ztest tests**:
272+
a Test Scenario identifier from the corresponding ``testcase.yaml`` file,
273+
a Ztest suite name, and a Ztest test name:
274+
``<Test Scenario identifier>.<Ztest suite name>.<Ztest test name>``
275+
276+
* **Standalone tests and samples**:
277+
a Test Scenario identifier from the corresponding ``testcase.yaml`` (or
278+
``sample.yaml``) file where the last section signifies the standalone
279+
Test Case name, for example: ``debug.coredump.logging_backend``.
277280

278281

279282
The following is an example test configuration with a few options that are
@@ -316,12 +319,10 @@ related to the sample and what is being demonstrated:
316319
tags: tests
317320
min_ram: 16
318321
319-
The full canonical name for each test scenario is:``<path to test application>/<test scenario identifier>``
320-
321-
A test scenario entry is a a block or entry starting with test scenario
322-
identifier in the YAML files.
322+
A Test Scenario entry in the ``tests:`` YAML dictionary has its Test Scenario
323+
identifier as a key.
323324

324-
Each test scenario entry in the test application configuration can define the
325+
Each Test Scenario entry in the Test Application configuration can define the
325326
following key/value pairs:
326327

327328
.. _test_config_args:

0 commit comments

Comments
 (0)