docs: Migrate existing public docs into readthedocs and git

Author: alhendrickson

docs/index.md

@@ -1,15 +1,26 @@
 
-# Cogstack Platform Toolit
+# Cogstack Documentation
 
-This project provides utilities for running Cogstack in production.
 
-- [CogStack Observability](observability/_index.md) 
+CogStack is composed of a range of adaptable modular interoperable tools which introduce tiered functionality which can be used for a variety of use-technologies:
+
+Centralise and lake clinical data including structured data i.e. observations, results, and unstructured data i.e. clinical narratives such as clinic letters, discharge and admission summaries and radiology reports also varying formats e.g. binary word docs, PDFs, images.
+
+Search and visualise millions of distinct data points in near-real-time – ‘unlocking’ capabilities that would otherwise have taken days or months previously.
+
+Natural Language Processing of clinical text to standardised clinical terminologies (SNOMED-CT) for interoperable clinical data combined with semantic context. This allows cohorting based on “find all patients with a heart attack”, regardless of how this has been referred to in the clinical text, such as “patient had myocardial infarct”, “MI“, “infarct of heart”, “cardiac infarct” and distinguishing “the patient’s father had a MI”.
+
+Deep phenotyping using NLP allows accelerated NHS clinical coding, disease registry submissions and advanced cohorting for observational studies.
+
+Population health dashboards for combining data from structured and text components of the electronic health record to track patient outcomes, enhance patient safety and improve patient care.
+
+Advanced analytics using generative AI for virtual trial emulation, high-dimensional patient or disease modelling and digital patient twins.
 
 ```{toctree}
 :hidden:
 
+overview/_index
 observability/_index
 
 ```
 
-

docs/overview/CogStack Documentation.md

@@ -0,0 +1,23 @@
+
+
+
+# CogStack Documentation
+
+# What is CogStack?
+
+CogStack is a lightweight distributed, fault tolerant database processing architecture and ecosystem, intended to make NLP processing and preprocessing easier in resource constrained environments. It comprises of multiple components, and has been designed to provide configurable data processing pipelines for working with EHR data. For the moment it mainly uses databases and files as the primary source of EHR data with the possibility of adding custom data connectors in the near future. It makes use of the [Apache-Nifi](https://nifi.apache.org/) framework in order to provide a fully configurable data processing pipeline with the goal of generating annotated JSON standardised schema files that can be readily indexed into [ElasticSearch](https://www.elastic.co/), stored as files or pushed back to a database.
+
+![](./attachments/54bb85e8-0428-4a56-a702-fd359272ed6e.png)
+
+The CogStack ecosystem has been developed as an open source project with the code available on GitHub: [https://github.com/CogStack/](https://github.com/CogStack/CogStack-Pipeline) .
+
+> [!IMPORTANT]
+> ### Tip
+>
+> Starting from version 1.2 CogStack is preferably being run as an ecosystem using a set of different microservices and deployed using [Docker Compose](https://docs.docker.com/compose/). The ready-to-use CogStack images are available to pull directly from the official Docker Hub under [cogstacksystems](https://hub.docker.com/u/cogstacksystems/) organisation. We’re actively pursuing running the stack in a K8s cluster also.
+
+# Why does this project exist?
+
+The CogStack consists of a range of technologies designed to to support modern, open source healthcare analytics within the NHS, and is chiefly comprised of the Elastic stack ([ElasticSearch](https://www.elastic.co/products/elasticsearch), [Kibana](https://www.elastic.co/products/kibana), etc.), [MedCAT](https://github.com/CogStack/MedCAT) (clinical natural language processing for named entity extraction and linking), clinical text [OCR](https://github.com/CogStack/ocr-service), clinical text de-identification. Since the processed EHR data can be represented and stored in databases or ElasticSearch, CogStack can be perfectly utilised as one of the solutions for integrating EHR data with other types of biomedical, -omics, wearables data, etc.
+
+---

docs/overview/CogStack Product Documentation.md

@@ -0,0 +1,18 @@
+
+
+
+# CogStack Product Documentation
+
+Welcome to the CogStack Documentation site.
+
+Any broad questions then please do reach out in our community space [here](https://discourse.cogstack.org/)
+
+Further in development projects are [here](https://github.com/orgs/CogStack/repositories)
+
+![](./attachments/43c14755-e565-4ae0-a0a3-ec6dc18a691c.png)
+
+|                                                                                                                                             |                                                                                                                             |
+|:--------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------|
+| [CogStack-Nifi](https://cogstack-nifi.readthedocs.io/en/latest/main.html) <br/>![](./attachments/36c0d23f-a632-4fbf-9f7c-6669e88bbd39.png) | [MedCAT](https://medcat.readthedocs.io/en/latest/) <br/>![](./attachments/09a8bb60-9864-41fa-be7b-cf9a9dc04498.png)  <br/> ||                                                                                                                                    |       |
+|:-----------------------------------------------------------------------------------------------------------------------------------|:------|
+| [MedCATTrainer](https://medcattrainer.readthedocs.io/en/latest/) <br/>![](./attachments/09a8bb60-9864-41fa-be7b-cf9a9dc04498.png) | <br/> |