feat: Create Deployment documentation and examples in Terraform (#13)

* ops: Create Examples and modules for deploying medcat to AWS, Azure and Openstack

* docs: Create documentation for the deployment examples

Author: alhendrickson

.devcontainer/devcontainer.json

@@ -5,8 +5,13 @@
 	// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
 	"image": "mcr.microsoft.com/devcontainers/base:jammy",
 	"features": {
-		"ghcr.io/devcontainers/features/docker-outside-of-docker:1": {},
-		"ghcr.io/devcontainers/features/python:1": {}
+		"ghcr.io/devcontainers/features/python:1": {},
+		"ghcr.io/devcontainers/features/docker-in-docker:2": {},
+		"ghcr.io/devcontainers/features/terraform:1": {},
+		"ghcr.io/devcontainers-extra/features/ansible:2": {},
+		"ghcr.io/devcontainers/features/kubectl-helm-minikube:1": {},
+		"ghcr.io/devcontainers/features/aws-cli:1": {},
+		"ghcr.io/devcontainers/features/azure-cli:1": {}
 	},
 
 	// Features to add to the dev container. More info: https://containers.dev/features.

.gitignore

@@ -2,6 +2,46 @@
 observability/examples/simple/observability-simple
 observability/examples/full/cogstack-observability
 _build
+**/.build/
+
+
+### Terraform Git Ignore
+# https://github.com/github/gitignore/blob/main/Terraform.gitignore
+
+# Local .terraform directories
+**/.terraform/
+
+# .tfstate files
+**/*.tfstate
+**/*.tfstate.*
+
+# Crash log files
+**/crash.log
+**/crash.*.log
+
+# Exclude all .tfvars files, which are likely to contain sensitive data
+**/*.tfvars
+**/*.tfvars.json
+
+# Ignore override files
+**/override.tf
+**/override.tf.json
+**/*_override.tf
+**/*_override.tf.json
+
+# Ignore transient lock info files created by terraform apply
+**/.terraform.tfstate.lock.info
+
+# Include override files you do wish to add to version control using negated pattern
+# !example_override.tf
+
+# Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan
+# example: *tfplan*
+
+# Ignore CLI configuration files
+**/.terraformrc
+**/terraform.rc
+
 
 # Python ignores
 # Byte-compiled / optimized / DLL files

.vscode/settings.json

@@ -0,0 +1,10 @@
+{
+    "files.exclude": {
+        "**/.terraform/": true,
+        "**/.terraform.lock.hcl": true,
+        "**/terraform.tfstate": true,
+        "**/terraform.tfstate.backup": true,
+        "**/terraform.tfstate.*.backup": true,
+        "**/.ansible/": true
+    }
+}

README.md

@@ -1,5 +1,10 @@
-# Cogstack Platform Toolkit
+# Cogstack Platform
 
 This repository contains tools and services used to support CogStack deployments
 
-See the latest documentation on [Readthedocs](https://cogstack-platform-toolkit.readthedocs.io/en/latest/observability/_index.html)
+See the latest documentation on [Readthedocs](https://docs.cogstack.org/en/latest/)
+
+## Project contents
+- Source for the official cogstack documentation. This git repo stores the top level documetation that hosts on https://docs.cogstack.org
+- CogStack deployment instructions and examples
+- CogStack platform tools eg Observability. 

deployment/terraform/examples/aws-kubernetes/.env.example

@@ -0,0 +1,3 @@
+export AWS_ACCESS_KEY_ID=
+export AWS_SECRET_ACCESS_KEY=
+export AWS_REGION=eu-west-1

deployment/terraform/examples/aws-kubernetes/.gitignore

@@ -0,0 +1,2 @@
+
+.env

deployment/terraform/examples/aws-kubernetes/README.md

@@ -0,0 +1,101 @@
+# AWS EKS Deployment
+
+This is an example deployment of CogStack in AWS. It will create publically accessible services, so is not suitable for production deployment.
+
+The recommended deployment in AWS is based on using Kubernetes through AWS EKS.
+
+This example will create a AWS EKS cluster, setup any necessary config, deploy CogStack to the cluster, and test that it is available.
+
+## Usage
+Deployment through terraform is carried out through two terraform commands, to handle the sequencing issues between making a k8s cluster and using it in AWS.
+
+### Requirements
+- Terraform - [Install Terraform](https://developer.hashicorp.com/terraform/install)
+- AWS Credentials for an account that can create and destroy resources. 
+
+
+### Steps
+
+### 1. Add Required Secrets for your env
+This readme uses environment variables for access:
+
+1. See the `.env.example` file for the required details.
+2. Create a file `.env` with those fields set for your account. 
+3. Execute `source .env` to set those environment variables
+
+If desired, see the official documentation for other ways to provide AWS credentials https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration
+
+### 2. Run Terraform
+Terraform is run on two modules for AWS, so we will run one terraform apply in one folder, then another terraform apply in a second folder. 
+
+Initial provisioning takes around 15 minutes.
+
+```bash
+# Set AWS credentials 
+source .env
+
+# Create AWS EKS infra
+cd eks-cluster
+terraform init
+terraform apply --auto-approve
+
+AWS_KUBECONFIG=$(terraform output -raw kubeconfig_file)
+
+# Deploy services to kubernetes
+cd ../kubernetes-deployment
+export TF_VAR_kubeconfig_file=$AWS_KUBECONFIG
+terraform init
+terraform apply --auto-approve
+```
+
+### 3. Accessing the CogStack Platform
+
+Once the deployment is complete and all services are running, you can access the CogStack platform and its components using the following URLs:
+
+```bash
+terraform output service_urls
+```
+
+
+### Optional - Destroy
+
+You can destroy the infra to save costs when it wont be used for a long time.
+
+Do note that there is an initial cost every time the EKS infrastructure is created, looks to be around $0.50 at time of writing.
+
+```bash
+cd ../kubernetes-deployment
+terraform destroy
+
+cd ../eks-cluster
+terraform destroy
+```
+
+
+## Optionally use the K8s cluster as normal with the CLI
+After setting up the cluster, it is possible to interact directly with it using the kubectl CLI
+
+The requirement is to get the KUBECONFIG file created by the terraform apply.
+
+```bash
+# Get KUBECONFIG
+cd eks-cluster
+AWS_KUBECONFIG=$(terraform output -raw kubeconfig_file)
+
+# SET KUBECONFIG
+export KUBECONFIG=${AWS_KUBECONFIG}
+```
+
+Note - alternatively you could use the AWS CLI to set your kubeconfig using `aws eks update-kubeconfig --name $(terraform output -raw cluster_name)`. 
+
+You can then interact with kubernetes via the CLI 
+
+```bash
+# Run Medcat service
+helm install my-medcat oci://registry-1.docker.io/cogstacksystems/medcat-service-helm --wait --timeout 10m0s
+
+# Create the ingress
+kubectl apply -f resources/ingress-medcat-service.yaml
+# Find public url
+kubectl get ingress
+```