You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: quality-checks.md
+17-15Lines changed: 17 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,41 +4,43 @@ This is part of a broader [quality framework](README.md)
4
4
5
5
# Summary
6
6
7
-
Quality checks are at the heart of good engineering and are essential for rapid and safe delivery of software changes. This page provides an index of the various quality checks described within our principles, patterns and practices.
7
+
Quality checks are at the heart of good engineering, and are essential for rapid and safe delivery of software changes. This page provides an index of the various quality checks described within our principles, patterns and practices.
8
8
9
9
# Usage
10
10
11
-
We recommend all projects should apply all of the applicable quality checks
11
+
All applicable quality checks should be applied. Not all checks are applicable in all contexts, for example accessibility testing is only applicable to applications with a user interface.
12
12
13
-
Not all checks are applicable in all contexts, for example accessibility testing is obviously not applicable to products without a user interface
13
+
The majority of these checks should be [automated](./patterns/automate-everything.md) via [continuous integration / continuous deployment](./practices/continuous-integration.md): the optimal sequencing of these checks within CI/CD pipelines will depend on the project's branching strategy, deployment strategy, etc.
14
+
15
+
All of these checks are important, even where their purpose overlaps with other checks in the list. For example, comprehensive functional testing could be achieved without unit testing, instead only using the other functional test types on this list - this would result in a very long-running and inefficient test suite, precluding fast feedback and impeding rapid and safe delivery of software changes. For further details please see [test practices](./practices/testing.md), and especially the [test pyramid](https://martinfowler.com/articles/practical-test-pyramid.html).
14
16
15
17
## RAG scale
16
18
17
-
We rate our projects against these checks as follows:
19
+
We rate our applications against each of these checks as follows:
18
20
19
-
* Green = check is applied frequently and consistently (typically via CI/CD), the output of the check is a quality gate (as opposed to just a warning / for information), and the tolerances for that quality gate (e.g. code coverage %) are agreed and understood
20
-
* Amber = check is applied, but not all conditions for green are met
21
-
* Red = check not in place
22
-
* N/A = check does not apply in the context
21
+
* Green = the quality check is applied frequently and consistently (in practice this typically means [automated](./patterns/automate-everything.md)via [continuous integration / continuous deployment](./practices/continuous-integration.md)), the output of the check is a quality gate (as opposed to just a warning / for information), and the tolerances for that quality gate (e.g. code coverage %) are agreed and understood
22
+
* Amber = the quality check is applied, but not all conditions for green are met - for example the check generates warnings that may or may not be acted on
23
+
* Red = the quality check is not applied
24
+
* N/A = the quality check is not applicable
23
25
24
26
# Details
25
27
26
28
| Quality check | Classification | Applicability | What it means | We we care | Tolerances for green | Endorsed tools / configuration | Further details |
27
29
|:---|:---|:---|:---|:---|:---|:---|:---|
28
-
| Unit tests | Functionality | Universal | Logic tests for blocks of code, e.g. methods |Fast & early feedback of logic issues| CI/CD builds fail if any tests fail | - |[Test practices](./practices/testing.md)|
30
+
| Unit tests | Functionality | Universal | Logic tests for individual blocks of code, e.g. individual methods |This is the quickest (to execute) type of functional test, so these are essential to achieve both rapid and thorough functional testing | CI/CD builds fail if any tests fail | - |[Test practices](./practices/testing.md)|
| API / contract tests | Functionality | Contextual ||||||
31
33
| UI tests | Functionality | Contextual ||||||
32
-
| Secret scanning | Security | Universal ||||||
34
+
| Secret scanning | Security | Universal |Check for secrets (e.g. passwords, IP addresses, etc) accidentally included in software code | This protects us against accidentally leaking secrets (in source code) which could compromise the security of the application | CI/CD builds fail if any unexpected secrets are detected | TBC | TBC|
| Performance tests | Resilience | Contextual |Check whether application performance is acceptable at different levels of load | Without this test, we don't know how load will affect the performance of the application||||
39
+
| Capacity tests | Resilience | Contextual |Identify the application's breaking point in terms of heavy load | Without this test, we don't know how much load the application can handle before the application breaks||||
40
+
| Stress tests | Resilience | Contextual |Check whether sudden spikes in load cause a problem | Without this test, we don't know if the application will fail under a sharp increase in load||||
41
+
| Soak tests | Resilience | Contextual |Check whether sustained heavy load causes a problem | Without this test, we don't know if application performance will suffer under prolonged heavy load||||
| Code coverage | Maintainability | Universal |The proportion of the application code which is executed (in this context: during testing) | The higher the code coverage, the more thorough the testing, and therefore the higher the likelihood of detecting functional issues early | For new code: 70%|||
0 commit comments