Fixing image links, ordering

We can (and should) use autogenerated sidebars for ease of maintenance. In order
to migrate to that, this change adds a directory structure that mirrors the old content's
static structure.

Note: rather than linking to the same file from multiple places, which is a bit confusing,
I've just included the file once in the first pillar it was found. As a result some pillars are empty
and other have fewer contents than the previous site. This is a feature, not a bug

Author: spikeheap

…tecture-pillars/automated_remediation.md …s/1-Reliability/automated_remediation.mddocs/architecture-pillars/automated_remediation.md renamed to docs/architecture-pillars/1-Reliability/automated_remediation.md docs/architecture-pillars/automated_remediation.md renamed to docs/architecture-pillars/1-Reliability/automated_remediation.md

@@ -1,10 +1,7 @@
----
-id: automated_remediation
-title: AWS Config for enabling automated remediation 
----
+# AWS Config for enabling automated remediation
 
 ### Context
-AWS Config is an AWS service that provides a method to assess if the configuration of AWS resources deployed in an account are compliant with a set of manually configured rules and allows us to keep track of all changes made. 
+AWS Config is an AWS service that provides a method to assess if the configuration of AWS resources deployed in an account are compliant with a set of manually configured rules and allows us to keep track of all changes made.
 
 AWS provides a set of predefined rules that can be used, but also has the option of creating custom ones. The rules that are being evaluated are chosen by the customer on a per account/per resource basis based on the service and organisation needs and identify if the configuration of an existing or newly provisioned resource is compliant with common best practices - for example, if encryption is enabled for DynamoDB tables.
 
@@ -13,7 +10,7 @@ AWS Config official documentation that details additional features can be found
 In Hackney, we are adopting several mechanisms to ensure that deployed resources are compliant with our preferred configuration, e.g. no database should be publicly accessible. Those include:
 - Following the [least privilege principle](./dev_least_principles.md), where engineers are unable to manually create AWS resources but must instead automate the creation via IaC (infrastructure as code) and CI/CD pipelines.
 - Adopting [Terraform-compliance](https://playbook.hackney.gov.uk/API-Playbook/terraform_compliance) security and compliance testing framework for performing pre-deployment  checks to ensure only compliant resources are deployed.
-- Using [serverless safeguards](https://playbook.hackney.gov.uk/API-Playbook/serverless_safegaurd)  to ensure AWS resources provisioned via the Serverless framework are also compliant. 
+- Using [serverless safeguards](https://playbook.hackney.gov.uk/API-Playbook/serverless_safegaurd)  to ensure AWS resources provisioned via the Serverless framework are also compliant.
 
 However, some of those must be set up on a per-project basis, thus not guaranteeing security risks prevention by the proposed mechanisms due to no current way to ensure 100% take up amongst project teams.
 
@@ -22,16 +19,16 @@ Enabling AWS Config with automated remediation can be done on account level, ens
 ### Vision
 - Have a defined list of compliance rules for commonly used AWS resources that is frequently reviewed and updated as and when required so that we have consistency for rules applied as well as a process to iterate that list when we change our use of existing or new AWS services.
 - Continuously monitor and detect non-compliant AWS resources so that we have visibility over potential security risks and performance, availability and cost impacting resources in our environments .
-- Automate remediation for breached rules so that our teams can focus on the other stages of a service delivery lifecycle and we have assurance that non-compliant resources would be corrected even if teams are unavailable to action a given compliance breach. 
+- Automate remediation for breached rules so that our teams can focus on the other stages of a service delivery lifecycle and we have assurance that non-compliant resources would be corrected even if teams are unavailable to action a given compliance breach.
 - Provide assurance for our security setup so that stakeholders have a strong confidence in the setup of our environments.
 - Ensure that resources provisioned are set up for performance, high availability and are cost optimised so that our services are more reliable and cost efficient.
 
 ### User needs
 As an **architect**, I want to ensure
 - that a solution’s design is implemented with the appropriate infrastructure resources configuration that is compliant with Hackney’s list of compliance requirements, which aim to promote better security, performance, reliability and availability at the lowest possible cost for a given service.
 
-As an **engineer**, I want 
-- a way to automate remediation of non-compliant AWS resources so that I can focus on the rest of the software lifecycle / platform activities. 
+As an **engineer**, I want
+- a way to automate remediation of non-compliant AWS resources so that I can focus on the rest of the software lifecycle / platform activities.
 
 As a **security analyst**, I want
 - assurance that services are configured as per agreed security requirements for infrastructure components.
@@ -41,7 +38,7 @@ As a **TDA member**, I want
 - to give assurance to Hackney senior stakeholders, staff and residents that services built are compliant and if audited, all criteria would be satisfied.
 
 ### Method
-Implement AWS Config rules and automated remediation, via Terraform, per account for the following types of resources (as part of the first iteration). 
+Implement AWS Config rules and automated remediation, via Terraform, per account for the following types of resources (as part of the first iteration).
 - AWS S3 [add terraform module link]
     - Automatically block public access
 - AWS RDS[add terraform module link]
@@ -58,11 +55,11 @@ Implement AWS Config rules and automated remediation, via Terraform, per account
 
 *The list of resources will be expanded following the trial of the tool. The initial list was prepared based on AWS resources commonly used for the development of our services.*
 
-## Considerations 
+## Considerations
 ### Cost
 As this is an AWS paid service and we have other compliance testing mechanisms in place (e.g. terraform-compliance), should we consider implementing AWS Config only for our Production environment?
 
 ### Implementation
-As a first iteration, this paper proposes to implement AWS Config with the set of rules and remediations already provided by AWS, instead of building custom ones. Should the tool work well for us, as well as if we identify a need for a custom rule/remediation, then that should be considered. 
+As a first iteration, this paper proposes to implement AWS Config with the set of rules and remediations already provided by AWS, instead of building custom ones. Should the tool work well for us, as well as if we identify a need for a custom rule/remediation, then that should be considered.
 
 Further to this, can we use Control Tower to automatically enable those rules for new and existing AWS accounts instead of using terraform?

…ture-pillars/core_resource_compliance.md …-Reliability/core_resource_compliance.mddocs/architecture-pillars/core_resource_compliance.md renamed to docs/architecture-pillars/1-Reliability/core_resource_compliance.md docs/architecture-pillars/core_resource_compliance.md renamed to docs/architecture-pillars/1-Reliability/core_resource_compliance.md

@@ -1,18 +1,15 @@
----
-id: core_resource_compliance
-title: Core AWS resources compliance checks 
----
+# Core AWS resources compliance checks
 
 ### Context
 At Hackney, we follow an infrastructure-as-code (IaC) approach and use Terraform to provision most of our AWS cloud resources. For our APIs, which are Lambda functions exposed via AWS API Gateway, we use the Serverless framework as it significantly speeds up the delivery and resource creation.  For more information please refer to [our playbook](https://playbook.hackney.gov.uk/API-Playbook/).
 
-From a Development perspective, each project manages its own Terraform files(or Serverless configuration) to provision resources for our microservices and frontend applications. Terraform is then applied automatically as part of the CI/CD pipeline workflow during deployment. 
+From a Development perspective, each project manages its own Terraform files(or Serverless configuration) to provision resources for our microservices and frontend applications. Terraform is then applied automatically as part of the CI/CD pipeline workflow during deployment.
 
-As Terraform files live in the same repository as the service, for which they are used to create cloud resources, we use our Pull Request process to identify any potential issues with changes to AWS resources or the configuration for adding new ones. Despite having a very thorough pull request process, there is still room for error if something gets missed during a review. 
+As Terraform files live in the same repository as the service, for which they are used to create cloud resources, we use our Pull Request process to identify any potential issues with changes to AWS resources or the configuration for adding new ones. Despite having a very thorough pull request process, there is still room for error if something gets missed during a review.
 
-To ensure we have security assurance in every step, we started using terraform-compliance -  a security and compliance test framework, and Serverless Safeguards to assess if changes are compliant with a predefined set of compliance rules and terminate deployment in case of a failure. 
+To ensure we have security assurance in every step, we started using terraform-compliance -  a security and compliance test framework, and Serverless Safeguards to assess if changes are compliant with a predefined set of compliance rules and terminate deployment in case of a failure.
 
-This document aims to outline the core compliance checks that **must** be performed for the various AWS resources provisioned as part of the Software Delivery Lifecycle **only** and not the wider platform. 
+This document aims to outline the core compliance checks that **must** be performed for the various AWS resources provisioned as part of the Software Delivery Lifecycle **only** and not the wider platform.
 
 
 ## Resources
@@ -24,7 +21,7 @@ Terraform compliance checks:
 
 ### API Gateway
 Serverless safeguards policies:
-1. Require Lambda authorizer for all API endpoints exposed by API Gateway with the exception of Swagger endpoints. 
+1. Require Lambda authorizer for all API endpoints exposed by API Gateway with the exception of Swagger endpoints.
     -  Example implementation [here](https://github.com/LBHackney-IT/asset-information-api/pull/51/files).
 2. Ensure cors is enabled.
 3. WAF is enabled.

…tecture-pillars/preferred_data_source.md …s/1-Reliability/preferred_data_source.mddocs/architecture-pillars/preferred_data_source.md renamed to docs/architecture-pillars/1-Reliability/preferred_data_source.md docs/architecture-pillars/preferred_data_source.md renamed to docs/architecture-pillars/1-Reliability/preferred_data_source.md

@@ -1,20 +1,17 @@
----
-id: preferred_data_source
-title: Preferred types of databases and when to use
----
+# Preferred types of databases and when to use
 
 ### Creating a new database
 
 **Before creating a new database**, please consult one of the Senior Engineers and/or confirm at the Data meetup if this type of data is stored elsewhere already.
 - Data might already exist and can be reused as per our approaches.
-- Existing data entities could potentially be re-iterated and expanded to include additional data properties, instead of creating a new data source making it less restrictive for reusability. 
+- Existing data entities could potentially be re-iterated and expanded to include additional data properties, instead of creating a new data source making it less restrictive for reusability.
 - Check our [SwaggerHub page](https://app.swaggerhub.com/organizations/Hackney) and [Developer Hub](https://developer-api.hackney.gov.uk/), which lists all of our APIs.
 
 **If a new data store is required:**
 - Perform an evaluation if SQL or NoSQL is more suitable for your project’s needs.
     - [Guidance provided further down in this document.](#choosing-the-right-type-of-database-technology)
-- Design the API that will interact with the data and present it at the Data Meetup as per our [API specifications assessment process.](https://playbook.hackney.gov.uk/api-specifications/assessment_process/) 
-- Use Terraform to provision the new database resource in AWS. 
+- Design the API that will interact with the data and present it at the Data Meetup as per our [API specifications assessment process.](https://playbook.hackney.gov.uk/api-specifications/assessment_process/)
+- Use Terraform to provision the new database resource in AWS.
     - Use one of the [Terraform common repository](https://github.com/LBHackney-IT/aws-hackney-common-terraform) templates (if applicable)
 
 
@@ -30,8 +27,8 @@ Assuming knowledge of the main differences between a SQL and a NoSQL database, t
 - *No: DynamoDB* - When our data structure changes and there is a need to have flexibility, NoSQL is a good choice, especially for continuously evolving data entities as part of agile development.
 
 **3. Do we need to support a lot of queries on different entity’s properties (*this question excludes search functionalities*)?**
-- *Yes: PostgreSQL* -  Queries on properties different  than Id and PartitionKey are anti-pattern for scaling out and so not suitable for NoSQL DB, those queries should be executed occasionally. 
-- *No: DynamoDB* 
+- *Yes: PostgreSQL* -  Queries on properties different  than Id and PartitionKey are anti-pattern for scaling out and so not suitable for NoSQL DB, those queries should be executed occasionally.
+- *No: DynamoDB*
 
 **4. Do we need low latency/sub-second data access? (this question excludes search functionalities)?**
 - *Yes: DynamoDB* - When there is a need for low latency data access, NoSQL tends to be really fast and in the order of ~10ms