You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: UNITTESTS/README.md
+43-38Lines changed: 43 additions & 38 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,27 +1,29 @@
1
1
## Unit testing
2
2
3
-
This document describes how to write and test unit tests for Arm Mbed OS. To prevent and solve problems, please see the [troubleshooting](#troubleshooting) section.
3
+
This document describes how to write and use unit tests for Arm Mbed OS.
4
4
5
5
### Introduction
6
6
7
-
Unit tests test code in small sections on a host machine. Unlike other testing tools, unit testing doesn't require embedded hardware, and it doesn't need to build the full operating system. Because of this, unit testing can result in faster tests than other testing tools. Unit testing takes place in a build environment where we test each C or C++ class or module in isolation. This means we build test suites into separate test binaries and stub all access outside to remove dependencies on any specific embedded hardware or software combination. This allows us to complete the testing using native compilers on the build machine.
7
+
Unit tests test code in small sections on a host machine. Unlike other testing tools, unit testing doesn't require embedded hardware or need to build a full operating system. Because of this, unit testing can result in faster tests than other tools. Unit testing happens in a build environment where you test each C or C++ class or module in isolation. Build test suites into separate test binaries and stub all access outside to remove dependencies on any specific embedded hardware or software combination. This allows you to complete tests using native compilers on the build machine.
8
8
9
9
### Prerequisites
10
10
11
-
Please install the following dependencies to use Mbed OS unit testing.
11
+
Please install the following dependencies to use Mbed OS unit testing:
12
12
13
-
- GNU toolchains.
14
-
- GCC 6 or later. We recommend you use MinGW-W64 on Windows, but any Windows port of the above GCC versions works. Default compilers can be used on Mac OS instead of GCC to shorten build times, but code coverage results can then differ.
15
-
- CMake 3.0 or newer.
16
-
- Python 2.7.x, 3.5 or newer.
17
-
- Pip 10.0 or newer.
18
-
- Gcovr 4.1 or newer.
19
-
- Arm Mbed CLI 1.8.0 or newer.
13
+
* GNU toolchains.
14
+
* GCC 6 or later. We recommend you use MinGW-W64 on Windows, but any Windows port of the above GCC versions works. Default compilers can be used on Mac OS instead of GCC to shorten build times, but code coverage results can differ.
15
+
* CMake 3.0 or newer.
16
+
* Python 2.7.x, 3.5 or newer.
17
+
* Pip 10.0 or newer.
18
+
* Gcovr 4.1 or newer.
19
+
* Arm Mbed CLI 1.8.0 or newer.
20
20
21
21
Detailed instructions for supported operating systems are below.
22
22
23
23
#### Installing dependencies on Debian or Ubuntu
24
24
25
+
In a terminal window:
26
+
25
27
1.`sudo apt-get -y install build-essential cmake`
26
28
1. Install Python and Pip with:
27
29
@@ -34,6 +36,8 @@ Detailed instructions for supported operating systems are below.
34
36
35
37
#### Installing dependencies on macOS
36
38
39
+
In a terminal window:
40
+
37
41
1. Install [Homebrew](https://brew.sh/).
38
42
1. Install Xcode Command Line Tools with `xcode-select --install`.
39
43
1. Install CMake with: `brew install cmake`.
@@ -49,6 +53,8 @@ Detailed instructions for supported operating systems are below.
49
53
50
54
#### Installing dependencies on Windows
51
55
56
+
In a terminal window:
57
+
52
58
1. Download and install MinGW-W64 from [SourceForge](https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/).
53
59
1. Download CMake binaries from https://cmake.org/download/, and run the installer.
54
60
1. Download Python 2.7 or Python 3 from https://www.python.org/getit/, and run the installer.
@@ -57,42 +63,42 @@ Detailed instructions for supported operating systems are below.
57
63
58
64
### Test code structure
59
65
60
-
Unit tests are located in the Mbed OS repository under the `UNITTESTS` folder. We recommend unit test files use an identical directory path to the file under test. This makes it easier to find unit tests for a particular class or a module. For example, if the file under test is `some/example/path/ClassName.cpp`, then all the test files are in the `UNITTESTS/some/example/path/ClassName` directory. Each test suite needs to have its own `unittest.cmake` file for test configuration.
66
+
Find unit tests in the Mbed OS repository under the `UNITTESTS` folder. We recommend unit test files use an identical directory path as the file under test. This makes it easier to find unit tests for a particular class or a module. For example, if the file you're testing is `some/example/path/ClassName.cpp`, then all the test files are in the `UNITTESTS/some/example/path/ClassName` directory. Each test suite needs to have its own `unittest.cmake` file for test configuration.
61
67
62
-
All the class stubs should be located in the `UNITTESTS/stubs` directory. A single stub class can be used by multiple test suites and therefore should follow the naming convention of `ClassName_stub.cpp` for the source file and `ClassName_stub.h` for the header file. Use the actual header files for the unit tests and try not to stub headers if possible. The stubbed headers reside in the `UNITTESTS/target_h` directory.
68
+
All the class stubs should be located in the `UNITTESTS/stubs` directory. A single stub class can be used by multiple test suites and should follow the naming convention `ClassName_stub.cpp` for the source file, and `ClassName_stub.h` for the header file. Use the actual header files for the unit tests, and don't stub headers if possible. The stubbed headers reside in the `UNITTESTS/target_h` directory.
63
69
64
70
#### Test discovery
65
71
66
-
Registering unit tests for running is automatic, and the test runner handles registration. However, test files are not automatically assigned to be built. We build unit tests by using a separate build system, which searches for unit tests under the `UNITTESTS` directory.
72
+
Registering unit tests to run happens automatically, and the test runner handles registration. However, test files do not automatically build. Build unit tests with a separate system that searches for unit tests under the `UNITTESTS` directory.
67
73
68
-
For the build system to find and build any test suite automatically, you must include a unit test configuration file named `unittest.cmake` for each unit test suite. This configuration file lists all the source files required for the test build.
74
+
For the build system to find and build any test suite automatically, include a unit test configuration file named `unittest.cmake` for each unit test suite. This configuration file lists all the source files required for the test build.
69
75
70
76
#### Test names
71
77
72
78
The build system automatically generates names of test suites. The name is constructed by taking a relative file path from the UNITTESTS directory to the test directory and replacing path separators with dashes. For example, the test suite name for `some/example/path/ClassName.cpp` is `some-example-path-ClassName`. Suite names are used when deciding which test suites to run.
73
79
74
80
### Unit testing with Mbed CLI
75
81
76
-
Mbed CLI supports unit tests through `mbed test --unittests` command. For information on using Mbed CLI, please see the [CLI documentation](https://os.mbed.com/docs/latest/tools/developing-arm-mbed-cli.html).
82
+
Mbed CLI supports unit tests through the `mbed test --unittests` command. For information on using Mbed CLI, please see the [CLI documentation](https://os.mbed.com/docs/latest/tools/developing-arm-mbed-cli.html).
77
83
78
84
### Writing unit tests
79
85
80
-
A unit tests suite consists of one or more test cases. The test cases should cover all the functions in a class under test. All the external dependencies are stubbed including the other classes in the same module. Avoid stubbing header files. Finally, analyze the code coverage to ensure all the code is tested and no dead code is found.
86
+
A unit tests suite consists of one or more test cases. The test cases should cover all the functions in a class under test. All the external dependencies are stubbed including the other classes in the same module. Avoid stubbing header files. Finally, analyze code coverage to ensure all code is tested, and no dead code is found.
81
87
82
-
Please see the [documentation for Google Test](https://github.com/google/googletest/blob/master/googletest/docs/primer.md) to learn how to write unit tests using the framework. See the [documentation for Google Mock](https://github.com/google/googletest/blob/master/googlemock/docs/Documentation.md) if you want to write and use C++ mock classes instead of stubs.
88
+
Please see the [documentation for Google Test](https://github.com/google/googletest/blob/master/googletest/docs/primer.md) to learn how to write unit tests using its framework. See the [documentation for Google Mock](https://github.com/google/googletest/blob/master/googlemock/docs/Documentation.md) if you want to write and use C++ mock classes instead of stubs.
83
89
84
90
#### Test suite configuration
85
91
86
92
Create two files in the test directory for each test suite:
87
93
88
-
- Unit test source file (`test_ClassName.cpp`).
89
-
- Unit test configuration file (`unittest.cmake`).
94
+
* Unit test source file (`test_ClassName.cpp`).
95
+
* Unit test configuration file (`unittest.cmake`).
90
96
91
-
List all the files required for the build in the `unittest.cmake` file. You will need to list the file paths relative to the `UNITTESTS` folder. Use the following variables to list the source files and include paths:
97
+
List all the required files for the build in the `unittest.cmake` file with paths relative to the `UNITTESTS` folder. Use the following variables to list the source files and include paths:
92
98
93
-
-**unittest-includes** - List of header include paths. You can use this to extend or overwrite default paths listed in `UNITTESTS/CMakeLists.txt`.
94
-
-**unittest-sources** - List of files under test.
95
-
-**unittest-test-sources** - List of test sources and stubs.
99
+
***unittest-includes**: List of header include paths. You can use this to extend or overwrite default paths listed in `UNITTESTS/CMakeLists.txt`.
100
+
***unittest-sources**: List of files under test.
101
+
***unittest-test-sources**: List of test sources and stubs.
96
102
97
103
You can also set custom compiler flags and other configurations supported by CMake in `unittest.cmake`.
98
104
@@ -114,7 +120,7 @@ With the following steps, you can write a simple unit test. In this example, `rt
114
120
rtos/Semaphore/test_Semaphore.cpp
115
121
)
116
122
```
117
-
1. Stub all external dependencies. Create stubs `UNITTESTS/stubs/mbed_assert_stub.c` and `UNITTESTS/stubs/Kernel_stub.cpp` if they do not exist.
123
+
1. Stub all external dependencies. Create stubs `UNITTESTS/stubs/mbed_assert_stub.c` and `UNITTESTS/stubs/Kernel_stub.cpp` if they don't already exist.
118
124
1. Update header stubs with any missing type or function declarations.
119
125
1. Create a test source file `UNITTESTS/rtos/Semaphore/test_Semaphore.cpp` with the following content:
120
126
@@ -177,29 +183,28 @@ Use Mbed CLI to build and run unit tests. For advanced use, you can run CMake an
177
183
1. Create a build directory `mkdir UNITTESTS/build`.
178
184
1. Move to the build directory `cd UNITTESTS/build`.
179
185
1. Run CMake using a relative path to `UNITTESTS` folder as the argument. So from `UNITTESTS/build` use `cmake ..`:
180
-
- Add `-g [generator]` if generating other than Unix Makefiles such in case of MinGW use `-g "MinGW Makefiles"`.
181
-
- Add `-DCMAKE_MAKE_PROGRAM=<value>`, `-DCMAKE_CXX_COMPILER=<value>` and `-DCMAKE_C_COMPILER=<value>` to use a specific Make program and compilers.
182
-
- Add `-DCMAKE_BUILD_TYPE=Debug`to build a debug build.
183
-
- Add `-DCOVERAGE=True` to add coverage compiler flags.
184
-
- See the [CMake manual](https://cmake.org/cmake/help/v3.0/manual/cmake.1.html) for more information.
185
-
1. Run a Make program to build the tests.
186
+
* Add `-g [generator]` if generating other than Unix Makefiles such in case of MinGW use `-g "MinGW Makefiles"`.
187
+
* Add `-DCMAKE_MAKE_PROGRAM=<value>`, `-DCMAKE_CXX_COMPILER=<value>` and `-DCMAKE_C_COMPILER=<value>` to use a specific Make program and compilers.
188
+
* Add `-DCMAKE_BUILD_TYPE=Debug`for a debug build.
189
+
* Add `-DCOVERAGE=True` to add coverage compiler flags.
190
+
* See the [CMake manual](https://cmake.org/cmake/help/v3.0/manual/cmake.1.html) for more information.
191
+
1. Run a Make program to build tests.
186
192
187
193
#### Run tests directly with CTest
188
194
189
-
Run a test binary in the build directory to run a unit test suite. To run multiple test suites at once, use CTest test runner. Run CTest with `ctest`. Add `-v` to get results for each test case. See the [CTest manual](https://cmake.org/cmake/help/v3.0/manual/ctest.1.html) for more information.
195
+
Run a test binary in the build directory to run a unit test suite. To run multiple test suites at once, use the CTest test runner. Run CTest with `ctest`. Add `-v` to get results for each test case. See the [CTest manual](https://cmake.org/cmake/help/v3.0/manual/ctest.1.html) for more information.
190
196
191
197
#### Run tests with GUI test runner
192
198
193
-
1. Install *gtest-runner* using the [documentation](https://github.com/nholthaus/gtest-runner).
194
-
1. Run *gtest-runner*
195
-
1. Add test executables into the list.
196
-
1. Run them.
199
+
1. Install `gtest-runner` according to the [documentation](https://github.com/nholthaus/gtest-runner).
200
+
1. Run `gtest-runner`.
201
+
1. Add test executables into the list and run.
197
202
198
203
### Debugging
199
204
200
205
1. Use Mbed CLI to build a debug build. For advanced use, run CMake directly with `-DCMAKE_BUILD_TYPE=Debug`, and then run a Make program.
201
206
1. Run GDB with a test executable as an argument to debug unit tests.
202
-
1. Run tests with Valgrind to analyze test memory profile.
207
+
1. Run tests with Valgrind to analyze the test memory profile.
203
208
204
209
### Get code coverage
205
210
@@ -216,9 +221,9 @@ Use Mbed CLI to generate code coverage reports. For advanced use, follow these s
216
221
***Solution**: Delete the build directory. Make sure that CMake, g++, GCC and a Make program can be found in the path and are correct versions.
217
222
218
223
**Problem:** (Windows) Virus protection identifies files generated by CMake as malicious and quarantines the files.
219
-
***Solution**: Restore the falsepositive files from the quarantine.
224
+
***Solution**: Restore false-positive files from the quarantine.
220
225
221
-
**Problem:** (Windows) Git with shell installation adds sh.exe to the path and then CMake throws an error: sh.exe was found in your PATH. For MinGW make to work correctly sh.exe must NOT be in your path.
226
+
**Problem:** (Windows) Git with shell installation adds sh.exe to the path and then CMake throws an error: sh.exe was found in your PATH. For MinGW make to work correctly, sh.exe must NOT be in your path.
222
227
***Solution**: Remove sh.exe from the system path.
223
228
224
229
**Problem:** (Mac OS) CMake compiler check fails on Mac OS Mojave when using GCC-8.
0 commit comments