Skip to content

Commit a101d7a

Browse files
bridadanbridadan
committed
Updating mbed test documentation
1 parent 062f8f9 commit a101d7a

File tree

2 files changed

+111
-23
lines changed

2 files changed

+111
-23
lines changed

‎README.md

Lines changed: 110 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,10 @@ This document covers the installation and usage of *mbed CLI*.
3030
6. [Automating toolchain and target selection](#automating-toolchain-and-target-selection)
3131
1. [Exporting to desktop IDEs](#exporting-to-desktop-ides)
3232
1. [Testing](#testing)
33+
1. [Finding available tests](#finding-available-tests)
34+
2. [Change the test action](#change-the-test-action)
35+
3. [Limiting the test scope](#limiting-the-test-scope)
36+
4. [Test directory structure](#test-directory-structure)
3337
1. [Known limitations](#known-limitations)
3438

3539
## Installation
@@ -388,10 +392,10 @@ Image: .build/K64F/GCC_ARM/mbed-os-program.bin
388392

389393
The arguments to *compile* are:
390394

391-
* `-m <MCU>` to select a target for the compilation. At the moment, the only supported value for `mcu` is `K64F` (for the FRDM_K64F board).
392-
* `-t <TOOLCHAIN>` to select a toolchain, where `toolchain` can be either ARM (armcc compiler) or GCC_ARM (GNU ARM Embedded).
393-
* `--source <SOURCE>` to select the source directory. Default is `.` (the current dir). You can specify multiple source locations, even outside the program tree.
394-
* `--build <BUILD>` to select the build directory. Default: `.build/ inside your program
395+
* `-m <MCU>` to select a target
396+
* `-t <TOOLCHAIN>` to select a toolchain, where `toolchain` can be either `ARM` (armcc compiler), `GCC_ARM` (GNU ARM Embedded), or `IAR` (IAR Embedded Workbench for ARM)
397+
* `--source <SOURCE>` to select the source directory. Default is `.` (the current dir). You can specify multiple source locations, even outside the program tree
398+
* `--build <BUILD>` to select the build directory. Default: `.build/` inside your program
395399
* `--library` to compile the code as a [static .a/.ar library](#compiling-static-libraries)
396400
* `--config` to inspect the run-time compile configuration (see below)
397401
* `-S` or `--supported` shows supported matrix of targets and toolchains
@@ -504,31 +508,119 @@ A ``.uvproj`` file is created in the projectfiles/uvision folder. You can open t
504508

505509
### Testing
506510

507-
mbed OS comes with a comprehensive set of test tools (see blah.blah.com), and you can use *mbed CLI* to manage and execute tests for you. Use `mbed test --list-compile` to list the tests available:
511+
Use the `test` command to compile and run tests:
508512

509-
#### Listing and compiling tests
513+
```
514+
$ mbed test -m K64F -t GCC_ARM
515+
Building library mbed-build (K64F, GCC_ARM)
516+
Building project GCC_ARM to TESTS-unit-myclass (K64F, GCC_ARM)
517+
Compile: main.cpp
518+
Link: TESTS-unit-myclass
519+
Elf2Bin: TESTS-unit-myclass
520+
+-----------+-------+-------+------+
521+
| Module | .text | .data | .bss |
522+
+-----------+-------+-------+------+
523+
| Fill | 74 | 0 | 2092 |
524+
| Misc | 47039 | 204 | 4272 |
525+
| Subtotals | 47113 | 204 | 6364 |
526+
+-----------+-------+-------+------+
527+
Static RAM memory (data + bss): 6568
528+
Heap: 65540
529+
Stack: 32768
530+
Total RAM memory (data + bss + heap + stack): 104876
531+
Total Flash memory (text + data + misc): 48357
532+
Image: .build\tests\K64F\GCC_ARM\TESTS\mbedmicro-rtos-mbed\mutex\TESTS-unit-myclass.bin
533+
...[SNIP]...
534+
mbedgt: test suite report:
535+
+--------------+---------------+---------------------------------+--------+--------------------+-------------+
536+
| target | platform_name | test suite | result | elapsed_time (sec) | copy_method |
537+
+--------------+---------------+---------------------------------+--------+--------------------+-------------+
538+
| K64F-GCC_ARM | K64F | TESTS-unit-myclass | OK | 21.09 | shell |
539+
+--------------+---------------+---------------------------------+--------+--------------------+-------------+
540+
mbedgt: test suite results: 1 OK
541+
mbedgt: test case report:
542+
+--------------+---------------+---------------------------------+---------------------------------+--------+--------+--------+--------------------+
543+
| target | platform_name | test suite | test case | passed | failed | result | elapsed_time (sec) |
544+
+--------------+---------------+---------------------------------+---------------------------------+--------+--------+--------+--------------------+
545+
| K64F-GCC_ARM | K64F | TESTS-unit-myclass | TESTS-unit-myclass1 | 1 | 0 | OK | 5.00 |
546+
| K64F-GCC_ARM | K64F | TESTS-unit-myclass | TESTS-unit-myclass2 | 1 | 0 | OK | 5.00 |
547+
| K64F-GCC_ARM | K64F | TESTS-unit-myclass | TESTS-unit-myclass3 | 1 | 0 | OK | 5.00 |
548+
+--------------+---------------+---------------------------------+---------------------------------+--------+--------+--------+--------------------+
549+
mbedgt: test case results: 3 OK
550+
mbedgt: completed in 21.28 sec
551+
```
552+
553+
The arguments to `test` are:
554+
555+
* `-m <MCU>` to select a target for the compilation.
556+
* `-t <TOOLCHAIN>` to select a toolchain, where `toolchain` can be either `ARM` (armcc compiler), `GCC_ARM` (GNU ARM Embedded), or `IAR` (IAR Embedded Workbench for ARM)
557+
* `--compile-list` to list all the tests that can be built
558+
* `--run-list` to list all the tests that can be ran (they must be built first)
559+
* `--compile` to only compile the tests
560+
* `--run` to only run the tests
561+
* `-n <TESTS_BY_NAME>` to limit the tests built or ran to a comma separated list (ex. test1,test2,test3)
562+
* `--source <SOURCE>` to select the source directory. Default is `.` (the current dir). You can specify multiple source locations, even outside the program tree.
563+
* `--build <BUILD>` to select the build directory. Default: `.build/ inside your program
564+
* `-c or --clean` to clean the build directory before compiling
565+
* `--test-spec <TEST_SPEC>` to set the path for the test spec file used when building and running tests (the default path is the build directory)
566+
* `-v` or `--verbose` for verbose diagnostic output
567+
* `-vv` or `--very_verbose` for very verbose diagnostic output
568+
569+
The compiled binaries and test artifacts can be found in the `.build/tests/<TARGET>/<TOOLCHAIN>` directory of your program.
570+
571+
#### Finding available tests
510572

573+
You can find the tests that are available for **building** by using the `--compile-list` option:
511574

512575
```
513-
$ mbed test --list-compile
576+
$ mbed test --compile-list
514577
Test Case:
515-
Name: mbed-os-core-mbed-rtos-TESTS-mbed-rtos-signals
516-
Path: ./mbed-os/core/mbed-rtos/TESTS/mbed-rtos/signals
578+
Name: TESTS-functional-test1
579+
Path: .\TESTS\functional\test1
517580
Test Case:
518-
Name: mbed-os-core-mbed-rtos-TESTS-mbed-rtos-basic
519-
Path: ./mbed-os/core/mbed-rtos/TESTS/mbed-rtos/basic
581+
Name: TESTS-functional-test2
582+
Path: .\TESTS\functional\test2
520583
Test Case:
521-
Name: mbed-os-TESTS-integration-basic
522-
Path: ./mbed-os/TESTS/integration/basic
523-
...
584+
Name: TESTS-functional-test3
585+
Path: .\TESTS\functional\test3
524586
```
525587

526-
Tests are compiled by adding the argument ```--compile``` in the above compile command:
588+
You can find the tests that are available for **running** by using the `--run-list` option:
527589

528590
```
529-
$ mbed test --compile -t ARM -m K64F
591+
$ mbed test --run-list
592+
mbedgt: test specification file '.\.build/tests\K64F\ARM\test_spec.json' (specified with --test-spec option)
593+
mbedgt: using '.\.build/tests\K64F\ARM\test_spec.json' from current directory!
594+
mbedgt: available tests for built 'K64F-ARM', location '.\.build/tests\K64F\ARM'
595+
test 'TESTS-functional-test1'
596+
test 'TESTS-functional-test2'
597+
test 'TESTS-functional-test3'
530598
```
531599

600+
#### Change the test action
601+
602+
You can specify to only **build** the tests by using the `--compile` option:
603+
604+
```
605+
$ mbed test -m K64F -t GCC_ARM --compile
606+
```
607+
608+
You can specify to only **run** the tests by using the `--run` option:
609+
610+
```
611+
$ mbed test -m K64F -t GCC_ARM --run
612+
```
613+
614+
#### Limiting the test scope
615+
616+
You can limit the scope of the tests built and ran by using the `-n` option. This takes a comma separated list of test names as an argument:
617+
618+
```
619+
$ mbed test -m K64F -t GCC_ARM -n TESTS-functional-test1,TESTS-functional-test2
620+
```
621+
622+
#### Test directory structure
623+
532624
Test code exists in the following directory structure:
533625

534626
```
@@ -558,13 +650,9 @@ mbed-os-program
558650
| ....
559651
```
560652

561-
As shown above, tests exist inside ```TESTS\testgroup\testcase\``` directories. Please note that `TESTS` is a special upper case directory that is excluded from module sources while compiling. Any testing libraries can be put inside the ```frameworks``` directory. This is currently just a good convention, but in the future tests might be able to limit which libraries should and should not be included when compiling them.
562-
563-
Compiled test binaries are created in ```.build/<TARGET>/<TOOLCHAIN>/TestCase1.bin```
564-
565-
<span class="notes">**Note:** This feature does not work in application modules that contain ```main()```. This issue is being worked on in parallel. However, currently we don't have any module with ```main()``` and ```TESTS``` together. Hence it does not break any existing module.</span>
653+
As shown above, tests exist inside ```TESTS\testgroup\testcase\``` directories. Please note that `TESTS` is a special upper case directory that is excluded from module sources while compiling.
566654

567-
#### Listing and compiling tests
655+
<span class="notes">**Note:** This feature does not work in applications that contain a ```main``` function that is outside of a `TESTS` directory.</span>
568656

569657
## Known limitations
570658

‎mbed/mbed.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1938,7 +1938,7 @@ def compile(toolchain=None, mcu=None, source=False, build=False, compile_library
19381938
dict(name='--source', action='append', help='Source directory. Default: . (current dir)'),
19391939
dict(name='--build', help='Build directory. Default: .build/'),
19401940
dict(name=['-c', '--clean'], action='store_true', help='Clean the build directory before compiling'),
1941-
dict(name='--test-spec', dest="test_spec", help="Destination path for the test spec file used when running tests (only override if a test spec file that can be used by the Greentea automated test tool. The default is placed in the build directory"),
1941+
dict(name='--test-spec', dest="test_spec", help="Path used for the test spec file used when building and running tests (the default path is the build directory)"),
19421942
help='Find, build and run tests',
19431943
description=("Find, build, and run tests in a program and libraries"))
19441944
def test_(toolchain=None, mcu=None, compile_list=False, run_list=False, compile_only=False, run_only=False, tests_by_name=None, source=False, build=False, clean=False, test_spec=None):

0 commit comments

Comments
 (0)