|
9 | 9 | * [Governance as a side effect](../patterns/governance-side-effect.md)
|
10 | 10 | * [Quality Metrics](../quality-checks.md)
|
11 | 11 |
|
12 |
| -## General Testing Priniciples |
| 12 | +## General Testing Principles |
13 | 13 |
|
14 | 14 | * **Quality is the whole team's responsibility**
|
15 | 15 | * Education on testing and testing principles should be important to the whole team.
|
|
18 | 18 | * **Combining business knowledge with testing knowledge yields better quality outcomes**
|
19 | 19 | * Include business knowledge and critical thinking as part of assurance
|
20 | 20 | * Intrinsic knowledge and mindset of the team is key to driving quality outcomes
|
21 |
| - |
| 21 | + |
22 | 22 | * **Testing is prioritised based on risk**
|
23 | 23 | * A testing risk profile is defined and understood by the whole team, including the customer
|
24 | 24 | * Risk appetite should be worked across the whole team, including customers and/or users
|
25 | 25 | * Solution Assurance risks and Clinical Safety hazards must also be considered when prioritising risks
|
26 |
| - |
| 26 | + |
27 | 27 | * **Testing is context driven**
|
28 |
| - * Context should be considered when deciding on test techninques and tools to use. |
| 28 | + * Context should be considered when deciding on test techniques and tools to use. |
29 | 29 |
|
30 | 30 | * **Testing is assisted by automation**
|
31 | 31 | * Appreciate that not everything can be automated
|
|
35 | 35 | * Consider using testing tools to enhance other test techniques.
|
36 | 36 | * Eg. using record and play tools to aid exploratory UI testing
|
37 | 37 | * Eg. using API testing tools to aid exploratory API testing
|
38 |
| - |
| 38 | + |
39 | 39 | * **Testing should be continually improved**
|
40 | 40 |
|
41 | 41 | * **Testing is continuous**
|
42 | 42 | * Testing is a continuous activity, not a phase of delivery
|
43 |
| - * Testing can be performed in [all phases of a product's DevOps cycle](https://danashby.co.uk/2016/10/19/continuous-testing-in-devops/) |
44 |
| - |
45 |
| - |
| 43 | + * Testing can be performed in [all phases of a product's DevOps cycle](https://danashby.co.uk/2016/10/19/continuous-testing-in-devops/) and should be done as a collaborative exercise |
46 | 44 |
|
47 | 45 | ## Test Automation Theory
|
| 46 | + |
48 | 47 | * Test should be added at the most appropriate level of the [test pyramid](https://martinfowler.com/articles/practical-test-pyramid.html) — in general, the lower the better and avoid testing the same thing in multiple levels.
|
49 | 48 | * Consider using the [agile testing quadrant](https://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/) to help support a whole team approach around test planning and test automation.
|
50 | 49 |
|
51 |
| - |
52 | 50 | ## Guidelines for different types of testing
|
53 |
| -### Unit Testing Guidence |
| 51 | + |
| 52 | +### Unit Testing Guidance |
| 53 | + |
54 | 54 | * Tests act as documentation of the system's behaviour and should be clear enough to do this effectively.
|
55 | 55 | * Tests are written alongside or before whatever they are testing.
|
56 | 56 | * Tests and test suites must be executable from the command line via a single command.
|
57 | 57 | * Tests do not rely on other tests (i.e. each test can be executed in isolation, tests can be run concurrently and in any order).
|
58 |
| -* Tests only check a single condition. |
| 58 | +* Tests only check a single condition (single concept). |
59 | 59 | * Tests are idempotent (i.e. give the same result when re-run against the same code).
|
60 | 60 | * Test data is automatically generated when needed.
|
61 | 61 | * Do not add tests of no value (e.g. tests of default getters/setters) just to increase the coverage level — 100% coverage is not a good use of time.
|
62 | 62 | * Tests must pass before code can be merged.
|
63 | 63 | * Use stubs to test the behaviour of components in isolation — focus tests of multiple services on validating the interactions between services e.g. using consumer-driven contract testing.
|
64 |
| -### API Testing Guidence |
65 | 64 |
|
66 |
| -### UI Testing Guidence |
67 |
| -### Browser/OS Testing Guidence |
68 |
| -* Website user interfaces should be tested against an appropriate set of browsers and versions — in particular, no effort should be spent testing against unsupported browsers or unsupported versions of browsers. See supported browsers for [Staff](https://aalto.digital.nhs.uk/#/document/viewer/8c039de1-eec0-49cd-8af3-a97fed6a8bff?library=5464c07f-daf1-4eee-b9b6-22e6c4dfbbd0) and [Citizens](https://aalto.digital.nhs.uk/#/document/viewer/465e6d1b-f107-49eb-ad25-e72c0299d3a6?library=5464c07f-daf1-4eee-b9b6-22e6c4dfbbd0) |
| 65 | +### API Testing Guidance |
| 66 | + |
| 67 | +* When building an API start with an implementation of an automated contract test suite first. This can be supported by simple tools like Wiremock and Postman/Newman |
| 68 | +* Use contract tests to support technical and business documentation |
| 69 | +* Provide a web interface for exploratory testing. This usually comes out of the box with tooling that supports the OpenAPI specification. Although, this requires an environment to be set up or to be run on demand locally |
69 | 70 |
|
| 71 | +### UI Testing Guidance |
70 | 72 |
|
| 73 | +### Browser/OS Testing Guidance |
| 74 | + |
| 75 | +* Website user interfaces should be tested against an appropriate set of browsers and versions — in particular, no effort should be spent testing against unsupported browsers or unsupported versions of browsers. See supported browsers for [Staff](https://aalto.digital.nhs.uk/#/document/viewer/8c039de1-eec0-49cd-8af3-a97fed6a8bff?library=5464c07f-daf1-4eee-b9b6-22e6c4dfbbd0) and [Citizens](https://aalto.digital.nhs.uk/#/document/viewer/465e6d1b-f107-49eb-ad25-e72c0299d3a6?library=5464c07f-daf1-4eee-b9b6-22e6c4dfbbd0) |
71 | 76 |
|
72 | 77 | ## Other Tools To Consider
|
| 78 | + |
73 | 79 | * BDD tools to encode acceptance criteria in business terms as automated tests where appropriate.
|
74 |
| -* Chaos engineering / resilence testing e.g. using AWS Fault Injection Simulator (see [AWS FIS](../tools/aws-fis) for sample code) |
| 80 | +* Chaos engineering / resilience testing e.g. using AWS Fault Injection Simulator (see [AWS FIS](../tools/aws-fis) for sample code) |
75 | 81 | * Performance tools to check load, volume, soak and stress limits
|
76 | 82 |
|
77 |
| - |
78 |
| - |
79 | 83 | ## Further reading and resources
|
| 84 | + |
80 | 85 | * [Test Driven Development: By Example](https://learning.oreilly.com/library/view/test-driven-development/0321146530/)
|
81 | 86 | * [xUnit Test Patterns: Refactoring Test Code](https://learning.oreilly.com/library/view/xunit-test-patterns/9780131495050/)
|
82 |
| -* [Setting a foundation for successfull test automation](https://testautomationu.applitools.com/setting-a-foundation-for-successful-test-automation/), a short course by Angie Jones, who also has a talk on test automation strategy [based on game design](https://applitools.com/event/level-up-playing-the-automation-game/) |
| 87 | +* [Setting a foundation for successful test automation](https://testautomationu.applitools.com/setting-a-foundation-for-successful-test-automation/), a short course by Angie Jones, who also has a talk on test automation strategy [based on game design](https://applitools.com/event/level-up-playing-the-automation-game/) |
83 | 88 | * [Test Automation University](https://testautomationu.applitools.com/)
|
84 | 89 | * [Ministry of Testing](https://www.ministryoftesting.com/)
|
0 commit comments