Reorganize our documentation

docs/website/README.md

@@ -1,64 +1,72 @@
-The website for `ouroboros-consensus` is generated using [docusaurus](https://docusaurus.io/). If you want to build and view the generated website see Section [_Building the website_](#building-the-website).
+The website for `ouroboros-consensus` is generated using [Docusaurus](https://docusaurus.io/). To build and view the generated website, see [_Building the website_](#building-the-website).
 
-# Website content
+# Website Content
 
-The website source pages can be found in the [`contents`](./contents) directory. We divide our pages into two main categories:
+The front page is defined in the [`index.js`](src/components/HomepageFeatures/index.js) file. The main sections are configured in [`docusaurus.config.js`](./docusaurus.config.js).
 
-- "About Ouroboros", which contains high-level information about the Consensus protocols that the code in this repository implements. The pages for this category should be placed in the [`contents/about-ouroboros`](./contents/about-ouroboros) directory.
-- "For Developers", which contains information relevant for contributors to the Consensus code base. The pages for this category should be placed in the [`contents/for-developers`](./contents/for-developers) directory.
+The website's source pages are located in the [`contents`](./contents) directory. In accordance with the [Diátaxis framework](https://diataxis.fr/) for organizing technical documentation, we categorize our pages into four sections:
 
-The pages are written in Markdown format.
+- **Explanations** ([`contents/explanation`](./contents/explanation))
+- **Tutorials** ([`contents/tutorials`](./contents/tutorials))
+- **HOWTOs** ([`contents/howtos`](./contents/howtos))
+- **Reference** ([`contents/reference`](./contents/reference))
 
-## Adding a new page
+Refer to the [Diátaxis framework](https://diataxis.fr/) for guidance on the type of documentation that belongs in each section.
 
-To add a new page create a Markdown file inside the appropriate directory (either [`contents/about-ouroboros`](./contents/about-ouroboros) or [`contents/for-developers`](./contents/for-developers) depending on the category). Then edit the [`sidebars.js`](./sidebars.js) file and add a reference to the new page in the desired place inside the sidebar item's configuration. For instance, if you add a new page `for-developers/foo.md`, Docusaurus will generate a reference called `for-developers/foo`, which can be used to add an entry to the sidebar as follows:
+All pages are written in Markdown format.
+
+## Adding a New Page
+
+To add a new page, create a Markdown file inside the appropriate directory (e.g., [`contents/explanation`](./contents/explanation)), based on the Diátaxis category the page belongs to.
+
+Next, update [`sidebars.js`](./sidebars.js) and add a reference to the new page in the sidebar configuration. For example, if you add `reference/foo.md`, Docusaurus will generate a reference called `reference/foo`, which can be added to the sidebar as follows:
 
 ```javascript
 /** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */
 const sidebars = {
-
   ...
-
-  for_developers: [
+  reference: [
     { type: 'category',
-      label: 'For Developers',
+      label: 'Reference',
       items: [
-        'for-developers/index',
-        'for-developers/ComponentDiagram',
-        'for-developers/foo', // <--- Your new page
-        'for-developers/ChainSync',
+        'reference/index',
+        'reference/foo', // <--- Your new page
+        ...
       ]
-
+    }
+  ]
   ...
+};
 ```
 
-The order of the page in the `items` list determines the order in which this entry will appear in the sidebar. See [`sidebars.js`](./sidebars.js) for the rationale behind not autogenerating this.
+The order of entries in the `items` list determines their placement in the sidebar. Refer to [`sidebars.js`](./sidebars.js) for the rationale behind manual sidebar configuration.
 
 # Building the website
 
-To install the packages required to build the documentation site run:
+To install the required packages for the documentation site, run:
 
 ```
 $ yarn
 ```
 
-After the necessary packages are installed by the above command, the
-documentation site can be built by running:
+After installation, build the documentation site by running:
 
 ```
 $ yarn start
 ```
 
-This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
+This starts a local development server and opens a browser window. Most changes are reflected live without needing to restart the server.
 
 ## Build
 
+To generate static content:
+
 ```
 $ yarn build
 ```
 
-This command generates static content into the `build` directory and can be served using any static contents hosting service.
+This command outputs the static content into the build directory, which can be hosted using any static content hosting service.
 
 ## Deployment
 
-We deploy the website using a GitHub action. See [`.github/workflows/ci.yml`](../../.github/workflows/ci.yml).
+The website is deployed using a GitHub Action. See [`.github/workflows/ci.yml`](../../.github/workflows/ci.yml).

docs/website/contents/explanation/design-goals.md

@@ -0,0 +1,61 @@
+# Design Goals
+
+The components that make up `ouroboros-consensus` were designed with the following goals in mind.
+
+## Multiple Consensus Protocols
+
+The design must support different consensus algorithms, requiring abstraction over the specific choice of consensus protocol.
+From the project's inception, it was evident that multiple protocols would be required.
+The Byron era of Cardano utilizes the PBFT protocol, while the Shelley era transitioned to TPraos, and Praos has been used since the Babbage era.
+The consensus component must support not only the current era but also past eras, requiring the ability to compose protocols.
+Additionally, we must ensure support for integrating new consensus protocols.
+
+## Support for Multiple Ledgers
+
+Similar to the need for multiple consensus protocols, our implementation must support multiple ledger implementations.
+This is crucial for accommodating ledger changes as improvements are made.
+Abstracting over the ledger implementation enables the consensus layer to work with various ledgers.
+
+## Decoupling Consensus Protocol from Ledger
+
+The consensus protocol is designed to be independent of any specific ledger implementation.
+Since multiple ledgers (such as Shelley and its Shelley-based successors) can utilize the same consensus protocol (TPraos or Praos), the consensus protocol is defined based on what it expects or requires from the ledger rather than being tightly coupled to a specific one.
+This approach makes the consensus protocol abstract and reusable across different ledgers.
+
+## Testability
+
+Ensuring the thorough testability of the consensus layer is a critical design goal.
+As a core component of the Cardano Node, which manages the cryptocurrency, the consensus layer must adhere to strict correctness standards.
+Currently, we extensively employ property-based testing.
+Whenever possible, we should abstract over IO, enabling simulations of various failures (such as disk or network errors) to verify system recovery capabilities.
+Additionally, to leverage the property-based methodology, tests must be relatively inexpensive.
+The design should also support testing rare but expected scenarios (such as multiple slot leaders in Praos) by allowing overrides in protocol or ledger behavior at specific points.
+Furthermore, the system should facilitate the isolation testing of individual components.
+
+## Adaptability and Maintainability
+
+The consensus layer was developed as a replacement for a [previous implementation](https://github.com/input-output-hk/cardano-sl), with the immediate goal of transitioning from Byron/BFT to Shelley/Praos while supporting future ledger and protocol changes.
+This called for a flexible and adaptable design.
+Abstracting both the consensus algorithm and the ledger plays a crucial role in achieving this.
+Working with abstract interfaces prevents developers from making assumptions that may hold for one ledger but not others, avoiding costly fixes later.
+This abstraction also allows the consensus layer to be reused in other blockchain projects.
+Most importantly, an abstract design enables extensive testing with simpler mock ledgers, which are easier to set up and reason about compared to the complex real ledger.
+Abstraction is considered good engineering practice, enhancing clarity, reducing dependencies, and making the system easier to understand and maintain.
+
+## Composability
+
+Given the complexity and scale of the consensus layer codebase, it is essential to divide it into smaller, manageable components that can be understood and modified independently.
+Composability is a key technique employed to achieve this.
+A prime example is the Hard Fork Combinator (HFC), which enables the combination of different consensus protocols (such as BFT and Praos) and ledgers (such as Byron and Shelley) into a unified composite protocol or ledger for the hybrid chain.
+
+## Predictable Performance
+
+This goal ensures that node operators can configure nodes for "normal circumstances" without the network failing during infrequent but expected events.
+It aims to make node performance predictable, ensuring that the best-case scenario aligns with the worst-case scenario in terms of resource requirements—not only for security but also to maintain network stability with honest nodes.
+
+## Protection Against DoS Attacks
+
+The consensus layer must help safeguard the network against disruptions, making denial-of-service (DoS) attacks prohibitively expensive for adversaries.
+This involves design decisions that prevent attackers from easily causing a node to perform extensive, wasteful computations.
+For example, validating headers before downloading block bodies prevents attackers from forcing nodes to process potentially invalid blocks.
+The design often seeks a balance between the cost for an attacker to induce work and the cost for a node to defend against it.

docs/website/contents/explanation/index.md

@@ -0,0 +1,14 @@
+# Introduction
+
+Welcome to the documentation for the [`ouroboros-consensus`](https://github.com/IntersectMBO/ouroboros-consensus) repository.
+This repository houses the Haskell implementation of three crucial components utilized by the [`cardano-node`](https://github.com/input-output-hk/cardano-node): Consensus, Storage, and Mempool.
+- The [Consensus component](https://cardano-scaling.github.io/cardano-blueprint/consensus/index.html) implements the [Ouroboros](https://iohk.io/en/research/library/papers/ouroboros-a-provably-secure-proof-of-stake-blockchain-protocol/) family of Proof-of-Stake protocols.
+- The [Storage component](https://cardano-scaling.github.io/cardano-blueprint/storage/index.html) is responsible for providing efficient access to the blockchain data, as well as maintaining the current and recent past ledger states, and storing ledger state snapshots.
+- The [Mempool component](https://cardano-scaling.github.io/cardano-blueprint/mempool/index.html) serves as a buffer for valid transactions that are waiting to be included in a block. It is used by Consensus component when forging a block and by the Network layer's transaction submission mini-protocol when adding or getting transactions.
+
+A core design principle in the implementation of these components is the abstraction from specific ledger and protocol implementations.
+The aim is to decouple the consensus protocol from the ledger and to support multiple consensus algorithms and ledgers for improved adaptability and maintainability.
+This design allows different ledgers (like the Byron or Shelley ledgers) and different Ouroboros protocol instances (like Praos or TPraos) to be integrated into the abstract consensus framework.
+To reflect this design, the repository is structured into different sub-repositories.
+- The polymorphic implementations and abstract classes, which define the core consensus logic independently of specific ledger or protocol details, can be found in the [`ouroboros-consensus`](https://github.com/IntersectMBO/ouroboros-consensus/tree/main/ouroboros-consensus) sub-directory.
+- The Cardano specific instantiations, which provide the concrete implementations, reside in the [ouroboros-consensus-cardano](https://github.com/IntersectMBO/ouroboros-consensus/tree/main/ouroboros-consensus-cardano) sub-directory.