Chaos definition (#157)

Author: andyblundell

quality-checks.md

@@ -52,7 +52,7 @@ We recommend tracking progress on an Engineering Quality dashboard, for example:
 | Capacity tests         | Resilience      | Contextual    | Identify the application's breaking point in terms of heavy load                                                                                                                                                | Without this test, we don't know how much load the application can handle before the application breaks                                                                                                                                                    |                                                                                                                                                                          |                                                                               |                                                                                                               |
 | Stress tests           | Resilience      | Contextual    | Check whether sudden spikes in load cause a problem                                                                                                                                                             | Without this test, we don't know if the application will fail under a sharp increase in load                                                                                                                                                               |                                                                                                                                                                          |                                                                               |                                                                                                               |
 | Soak tests             | Resilience      | Contextual    | Check whether sustained heavy load causes a problem                                                                                                                                                             | Without this test, we don't know if application performance will suffer under prolonged heavy load                                                                                                                                                         |                                                                                                                                                                          |                                                                               |                                                                                                               |
-| Chaos tests            | Resilience      | Contextual    | Cause failures in a system to test the resiliency of that system and its environment, and to help the team to understand their MTTR                                                                                                                                                                                                                | Give the team confidence that failures in a given environment will not lead to unplanned downtime or a negative user experience<br/><br/>Ensures that the team has visibility (e.g. dashboards and alerts) to be able to identify issues                                                                                                                                                                                                                                                           | Builds fail if any test fails - note, these tests are slow, and are likely to be part of an infrequently-triggered (e.g. overnight) build<br/><br/>The tests cover whether the system self-heals, auto-scales, and alerts as expected                                                                                                                        |                                                                               |                                                                                                               |
+| Chaos tests            | Resilience      | Contextual    | Cause failures in a system to test the resiliency of that system and its environment, and our ability to respond to failures                                                                                                                                                                                                                | Give the team confidence that failures in a given environment will not lead to unplanned downtime or a negative user experience<br/><br/>Ensures that the team has visibility (e.g. dashboards and alerts) to be able to identify issues<br/><br/>Help the team to understand their mean time to recovery (MTTR) and to build muscle memory & confidence for recovery activities                                                                                                                                                                                                                                                           | Regular (at least every couple of months) game days, and:<br/><br/>Builds fail if any test fails - note, these tests are slow, and are likely to be part of an infrequently-triggered (e.g. overnight) build<br/><br/>The tests cover whether the system self-heals, auto-scales, and alerts as expected                                                                                                                        |                                                                               |                                                                                                               |
 | Code coverage          | Maintainability | Universal     | The proportion of the application code which is executed (in this context: during testing)                                                                                                                      | The higher the code coverage, the more thorough the testing, and therefore the higher the likelihood of detecting functional issues early                                                                                                                  | Must use SonarQube's default [rules, profiles and gateways](tools/sonarqube.md#default-quality-gates) <br/><br/> Build pipeline must fail if gateway test coverage is not met <br/><br/> For [new code](tools/sonarqube.md#definition-of-new-code), must meet coverage specified in default Sonarqube gateway. <br/> For legacy code, coverage amount can be initially lowered to deal with historic tech debt. But must have a plan for increasing coverage over time. | [SonarQube](tools/sonarqube.md) (in conjunction with testing frameworks)                                                                            |                                                                                                               |
 | Duplicate code scan    | Maintainability | Universal     | Check whether the same code is used in multiple places                                                                                                                                                                                                                | Duplicate code increases the cost and risk of making software changes and will increase exponentially over time | Must use SonarQube's default [rules, profiles and gateways](tools/sonarqube.md#default-quality-gates) <br/><br/> Build pipeline must fail if gateway is not met | [SonarQube](tools/sonarqube.md) |                                                                                                               |
 | Code smells scan       | Maintainability | Universal     | Check whether the software code adheres to the principles, patterns and practices of writing clean code                                                                                                         | The code is written once but read and executed many times. If the code is not clean, the cost and risk of making software changes both increase exponentially over time | Must use SonarQube's default [rules, profiles and gateways](tools/sonarqube.md#default-quality-gates) <br/><br/> Build pipeline must fail if gateway is not met | [SonarQube](tools/sonarqube.md) | [Clean Code: Smells and Heuristics](https://moderatemisbehaviour.github.io/clean-code-smells-and-heuristics/) |