Merge branch 'main' into rholling-SCS-docs

Signed-off-by: Kurt Garloff <[email protected]>

Author: garloff

.github/workflows/build.yml

@@ -22,7 +22,8 @@ jobs:
           key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
           restore-keys: |
             ${{ runner.os }}-node-
-      - name: Install dependencies
-        run: npm install
-      - name: build page
-        run: npm run build
+
+      - name: Install dependencies and build page
+        run: |
+          npm ci
+          npm run build

.gitignore

@@ -15,10 +15,12 @@
 /docs/04-operating-scs/components
 /docs/04-operating-scs/01-guides
 /docs/06-releases
+/docs/turnkey-solution
 /standards/*.md
 /standards/*/*.md
 /standards/*/*.mdx
 /standards/scs-*.yaml
+/user-docs/application-examples
 
 # Dependencies
 node_modules

.markdownlint-cli2.jsonc

@@ -55,6 +55,6 @@
     "markdownlint-rule-search-replace",
     "markdownlint-rule-relative-links"
   ],
-  "ignores": ["node_modules", ".github", "docs"],
+  "ignores": ["node_modules", ".github", "docs", "standards"],
   "globs": ["**/*.{md}"]
 }

README.md

@@ -26,3 +26,8 @@ CD in your Terminal to the root directory of the cloned repository. Install all
 npm install
 npm start
 ```
+
+## Linting problems
+
+The repository establishes commit hooks which check the files for correctness and style.
+Have a look at the [linting-guide](https://docs.scs.community/community/contribute/linting-guide/) to get detailed information.

community/contribute/adding-docs-guide.md

@@ -23,15 +23,15 @@ Your repository containing the documentation has to...
 
 The documentation files have to be in markdown format and...
 
-- comply [SCS licensing guidelines](https://github.com/SovereignCloudStack/docs/blob/main/community/github/dco-and-licenses.md)
+- comply [SCS licensing guidelines](https://github.com/SovereignCloudStack/docs/blob/main/community/license-considerations.md)
 - match our
   - [markdown file structure guideline](https://github.com/SovereignCloudStack/docs/blob/main/community/contribute/doc-files-structure-guide.md)
   - linting Rules
   - [styleguide](https://github.com/SovereignCloudStack/docs/blob/main/community/contribute/styleguide.md)
 
 ### Step 2 – Adding your repo to the docs.json
 
-File a Pull Request within the [docs-page](https://github.com/SovereignCloudStack/docs-page) repository and add your repo to the docs.package.json:
+File a Pull Request within the [docs](https://github.com/SovereignCloudStack/docs) repository and add your repo to the docs.package.json:
 
 ```json
 [

community/contribute/linting-guide.md

@@ -21,6 +21,13 @@ The markdownlint rules are defined in the configuration file `.markdownlint-cli2
 
 Additionally we use [markdownlint-rule-search-replace](https://github.com/OnkarRuikar/markdownlint-rule-search-replace) for fixing
 
+## Local Usage for development
+
+```bash
+npm run lint:md <file>
+npm run fix:md  <file>
+```
+
 ## Github Workflows
 
 There are two actions running on every Pull Request on the `main` branch.

docs.package.json

@@ -11,12 +11,6 @@
     "target": "docs/02-iaas/components",
     "label": ""
   },
-  {
-    "repo": "SovereignCloudStack/k8s-cluster-api-provider",
-    "source": "doc",
-    "target": "docs/03-container/components",
-    "label": "k8s-cluster-api-provider"
-  },
   {
     "repo": "SovereignCloudStack/cluster-stack-provider-openstack",
     "source": "docs",
@@ -35,6 +29,18 @@
     "target": "docs/02-iaas/",
     "label": ""
   },
+  {
+    "repo": "osism/osism.github.io",
+    "source": "docs/cloud-in-a-box",
+    "target": "docs/02-iaas/deployment-examples",
+    "label": ""
+  },
+  {
+    "repo": "osism/osism.github.io",
+    "source": "docs/testbed.mdx",
+    "target": "docs/02-iaas/deployment-examples",
+    "label": ""
+  },
   {
     "repo": "SovereignCloudStack/k8s-harbor",
     "source": "docs",
@@ -129,5 +135,17 @@
     "source": ["docs/*"],
     "target": "docs/03-container/components/cluster-stacks/components",
     "label": "cluster-stack-operator"
+  },
+  {
+    "repo": "SovereignCloudStack/hardware-landscape",
+    "source": ["documentation/overview.md"],
+    "target": "docs/turnkey-solution",
+    "label": ""
+  },
+  {
+    "repo": "SovereignCloudStack/opendesk-on-scs",
+    "source": "docs/*",
+    "target": "user-docs/application-examples",
+    "label": "opendesk-on-scs"
   }
 ]

docs/02-iaas/deployment-examples/artcodix/index.mdx

@@ -0,0 +1,161 @@
+# artcodix
+
+## Preface
+
+This document describes a possible environment setup for a pre-production or minimal production setup.
+In general hardware requirements can vary largely from environment to environment and this guide is not
+a hardware sizing guide nor the best placement solution of services for every setup. This guide intends to
+provide a starting point for a hardware based deployment of the SCS-IaaS reference implementation based on OSISM.
+
+## Node type definitions
+
+### Control Node
+
+A control node runs all or most of the openstack services, that are responsible for API-services and the corresponding
+runtimes. These nodes are necessary for any user to interact with the cloud and to keep the cloud in a managed state.
+However these nodes are usualy **not** running user virtual machines.
+Hence it is advisable to have the control nodes replicated. To have a RAFT-quorum three nodes are a good starting point.
+
+### Compute Node (HCI/no HCI)
+
+#### Not Hyperconverged Infrastructure (no HCI)
+
+Non HCI compute nodes are exclusively running user virtual machines. They are running no API-services, no storage daemons
+and no network routers, except for the necessary network infrastructure to connect virtual machines.
+
+#### Hyperconverged Infrastructure (HCI)
+
+HCI nodes generally run at least user virtual machines and storage daemons. It is possible to place networking services
+here as well but that is not considered good practice.
+
+#### No HCI / vs HCI
+
+Whether to use HCI nodes or not is in general not an easy question. For a getting started (pre production/smalles possible production)
+environment however, it is the most cost efficent option. Therefore we will continue with HCI nodes (compute + storage).
+
+### Storage Node
+
+A dedicated storage node runs only storage daemons. This can be necessary in larger deployments to protect the storage daemons from
+ressource starvation through user workloads.
+
+Not used in this setup.
+
+### Network Node
+
+A dedicated network node runs the routing infrastructure for user virtual machines that connects these machines with provider / external
+networks. In larger deployments these can be useful to enhance scaling and improve network performance.
+
+Not used in this setup.
+
+## Nodes in this deployment example
+
+As mentioned before we are running three dedicated control nodes. To be able to fully test an openstack environment it is
+recommended to run three compute nodes (HCI) as well. Technically you can get a setup running with just one compute node.
+See the following chapter (Use cases and validation) for more information.
+
+### Use cases and validation
+
+The setup described allows for the following use cases / test cases:
+
+- Highly available control plane
+  - Control plane failure toleration test (Database, RabbitMQ, Ceph Mons, Routers)
+- Highly available user virtual clusters (e.g. Kubernetes clusters)
+  - Compute host failure simulation
+- Host aggregates / compute node grouping
+- Host based storage replication (instead of OSD based)
+  - Fully replicated storage / storage high availability test
+
+### Control Node
+
+#### General requirements
+
+The control nodes do not run any user workloads. This means they are usually not sized as big as the compute nodes.
+Relevant metrics for control nodes are:
+
+- Fast and big enough discs. At least SATA-SSDs are recommended, NVMe will greatly improve the overall responsiveness.
+- A rather large amount of memory to house all the caches for databases and queues.
+- CPU performance should be average. A good compromise between amount of cores and speed should be used. However this is
+  the least important requirement on the list.
+
+#### Hardware recommendation
+
+The following server specs are just a starting point and can greatly vary between environments.
+
+Example:
+3x Dell R630/R640/R650 1HE Server
+
+- Dual 8 Core 3,00 GHz Intel/AMD
+- 128 GB RAM
+- 2x 3,84 TB NVMe in (Software-) RAID 1
+- 2x 10/25/40 GBit 2 Port SFP+/QSFP Network Cards
+
+### Compute Node (HCI)
+
+The compute nodes in this scenario run all the user virtual workloads **and** the storage infrastructure. To make sure
+we don't starve these nodes, they should be of decent size.
+
+> This setup takes local storage tests into consideration. The SCS-standards require certain flavors with very fast disc speed
+> to house customer kubernetes control planes (etcd). These speeds are usually not achievable with shared storage. If you don't
+> intend to test this scenario, you can skip the NVMe discs.
+
+#### Hardware recommendation
+
+The following server specs are just a starting point and can greatly vary between environments. The sizing of the nodes needs to fit
+the expected workloads (customer VMs).
+
+Example:
+3x Dell R730(xd)/R740(xd)/R750(xd)
+or
+3x Supermicro
+
+- Dual 16 Core 2,8 GHz Intel/AMD
+- 512 GB RAM
+- 2x 3,84 TB NVMe in (Software-) RAID 1 if you want to have local storage available (optional)
+
+For hyperconverged ceph osds:
+
+- 4x 10 TB HDD -> This leads to ~30 TB of available HDD storage (optional)
+- 4x 7,68 TB SSD -> This leads to ~25 TB of available SSD storage (optional)
+- 2x 10/25/40 GBit 2 Port SFP+/QSFP Network Cards
+
+## Network
+
+The network infrastructure can vary a lot from setup to setup. This guide does not intend to define the best networking solution
+for every cluster but rather give two possible scenarios.
+
+### Scenario A: Not recommended for production
+
+The smallest possible setup is just a single switch connected to all the nodes physically on one interface. The switch has to be
+VLAN enabled. Openstack recommends multiple isolated networks but the following are at least recommended to be split:
+
+- Out of Band network
+- Management networks
+- Storage backend network
+- Public / External network for virutal machines
+  If there is only one switch, these networks should all be defined as seperate VLANs. One of the networks can run in untagged default
+  VLAN 1.
+
+### Scenario B: Minimum recommended setup for small production environments
+
+The recommended setup uses two stacked switches connected in a LAG and at least three different physical network ports on each node.
+
+- Physical Network 1: VLANs for Public / External network for virutal machines, Management networks
+- Physical Network 2: Storage backend network
+- Physical Network 3: Out of Band network
+
+### Network adapters
+
+The out of band network does usually not need a lot of bandwith. Most modern servers come with 1Gbit/s adapters which are sufficient.
+For small test clusters, it might also be sufficient to use 1Gbit/s networks for the other two physical networks.
+For a minimum production cluster it is recommended to use the following:
+
+- Out of Band Network: 1Gbit/s
+- VLANs for Public / External network for virutal machines, Management networks: 10 / 25 Gbit/s
+- Storage backend network: 10 / 25 / 40 Gbit/s
+
+Whether you need a higher throughput for your storage backend services depends on your expected storage load. The faster the network
+the faster storage data can be replicated between nodes. This usually leads to improved performance and better/faster fault tolerance.
+
+## How to continue
+
+After implementing the recommended deployment example hardware, you can continue with the [deployment guide](https://docs.scs.community/docs/iaas/guides/deploy-guide/).

docs/03-container/index.md

@@ -18,26 +18,17 @@ The container layer within the Sovereign Cloud Stack (SCS) offers a robust solut
 ### Prerequisites and Requirements
 
 - Knowledge: Familiarity with Kubernetes, container orchestration, and basic cloud infrastructure principles is pivotal.
-- Software: The core software component is the K8s Cluster API Provider, crafted to function optimally on OpenStack environments. Although designed to run on the SCS IaaS layer, with minor configuration adjustments, it can operate on any OpenStack environment.
+- Software: The core software component are the Cluster Stacks based on Cluster API, crafted to function best on OpenStack environments. Although designed to run on the SCS IaaS layer, with minor configuration adjustments, it can operate on any OpenStack environment.
 - Hardware: Virtualization-enabled hardware capable of running OpenStack is essential if hosting the IaaS layer independently. For further details, refer to the IaaS layer documentation.
 
 ### Features
 
-- Automated Cluster Management: The K8s Cluster API Provider automates the process of creating, scaling, managing and updating Kubernetes clusters, thus significantly reducing the operational overhead.
+- Automated Cluster Management: The Cluster API automates the process of creating, scaling, managing and updating Kubernetes clusters, thus significantly reducing the operational overhead.
 - Standardized Operations: Upholding SCS standards across various clusters ensures operational consistency and reliability.
-- Integration with OpenStack: The K8s Cluster API Provider is tailored to work seamlessly with SCS IaaS (OpenStack), thus offering a unified platform for managing both containers and the underlying infrastructure.
-- Container Registry Integration: The container layer has an integrated container registry, facilitating easy management and deployment of container images.
-- Certificate Managment: The kubernetes clusters can optionaly include a certbot allowing for ease of deployment of public facing services out of the box.
-- Preconfigured ingress: Certificate Management: Optional inclusion of Certbot in Kubernetes clusters facilitates straightforward deployment of publicly accessible services.
-  Preconfigured Ingress: Kubernetes clusters come with a preconfigured Nginx ingress, designed with OpenStack in mind, providing a ready-to-use ingress solution with enhancements like out-of-the-box client source IP visibility.
+- Integration with OpenStack: The Cluster Stacks are tailored to work seamlessly with SCS IaaS (OpenStack), thus offering a unified platform for managing both containers and the underlying infrastructure.
+- Container Registry Integration: The container layer has an optional container registry, facilitating easy management and deployment of container images.
+- Cluster Addons: Cluster Stacks come with a small default set of workload applications needed to make the cluster usable, such as CNI plugin, CSI plugin and a cloud controller manager.
 
 ### Limitations
 
-- OpenStack Dependency: The current design primarily supports OpenStack environments, which could be a limitation for other infrastructure setups.
 - Serverless/Functions as a Service Support: Lack of direct support for serverless containers and Functions as a Service (FaaS) might require additional tools or platforms.
-
-### Current state and future Outlook
-
-The container layer has matured with multiple cloud providers now offering Kubernetes as a Service using this layer to manage a multitude of clusters. It follows a half-yearly release schedule to ensure security and up-to-date Kubernetes clusters, alongside providing backports for significant features into older versions.
-
-Looking ahead, a new version based on ClusterStacks is in the pipeline, currently in its Alpha state. This upcoming release aims to be backward compatible, facilitating smooth migration from existing setups, and further extending the capabilities of the SCS container layer.

docs/index.mdx

@@ -16,38 +16,54 @@ SCS is built, backed, and operated by an active open-source community worldwide.
 
 ## Use Cases and Deployment Examples
 
-### IaaS Layer
+### Virtualization (IaaS) Layer
 
-#### Quick Start with Cloud-In-A-Box
+The SCS IaaS Reference Implementation is based on [OSISM](https://osism.tech/).
 
-The fastest way to get in touch with SCS is to deploy a SCS cloud virtually. The Cloud-In-A-Box was built explicitly for this scenario. Check it out [here](/docs/iaas/guides/deploy-guide/examples/cloud-in-a-box)
+#### Quick Start with Cloud-in-a-Box
+
+You can do a single node installation for learning, testing or development purposes.
+The Cloud-in-a-Box configuration was built explicitly for this scenario.
+Check it out [here](/docs/iaas/deployment-examples/cloud-in-a-box)
+It comes with a complete set of services, even Ceph is part of it
+(despite of course not offering much redundancy on a single-node).
 
 #### Reference Implementation Testbed
 
+The fastest way to get in touch with SCS is to deploy a SCS cloud virtually.
+
 This means that you set up an SCS test installation including all the infrastructure
 pieces such as database, message queueing, ceph, monitoring and logging, IAM, the
 [OpenStack](https://openstack.org/) core services, and (soon) the Container layer
-on top of an existing IaaS platform.
+on top of an existing OpenStack IaaS platform, ideally one that allows for nested
+virtualization.
 
-The SCS IaaS reference implementation is based on [OSISM](https://osism.tech/). Read on the
-[OSISM testbed docs](https://docs.osism.de/testbed/) to learn how to get the
+Read the [testbed docs](/docs/iaas/deployment-examples/testbed) to learn how to get the
 testbed running. Please read carefully through the
-[deployment](https://docs.osism.de/testbed/deployment.html) section of the
+[deployment](/docs/iaas/deployment-examples/testbed#deployment) section of the
 manual.
 
-### Container Layer
+#### Examples for real deployments
 
-#### K8s Cluster API Provider
+[artcodix](https://artcodix.com/) has [shared](/docs/iaas/deployment-examples/artcodix/)
+some details on their production setup. The SCS team itself has created [extensive
+documentation](/docs/turnkey-solution/hardware-landscape/) including details on the
+used hardware.
+
+### Container Layer
 
-You can easily deploy the container layer on top of the testbed (or a production
-SCS cloud) checking out the code from
-[k8s-cluster-api-provider](https://github.com/SovereignCloudStack/k8s-cluster-api-provider/).
+The Reference Implementation (v2) for the container (Kubernetes-as-a-Service = KaaS) layer
+is provided by [Cluster Stacks](/docs/category/cluster-stacks)
+from [syself](https://syself.com/).
 
 #### Cluster Stacks
 
-With the Cluster Stacks, in the V2 KaaS reference implementation, we provide an opinionated optimized configuration of Kubernetes clusters. Through better packaging, integrated testing, and bundled configuration, SCS-based Kubernetes clusters provide easier individualization.  
-Throughout the R6 development cycle Cluster Stacks are taken from a technical preview to be [functional and available on top of the IaaS reference implementation](https://github.com/SovereignCloudStack/issues/milestone/8) as well to replace the V1 KaaS reference implementation [k8s-cluster-api-provider](https://github.com/SovereignCloudStack/k8s-cluster-api-provider/).  
-The Cluster Stacks can already be tried with the [demo](https://github.com/SovereignCloudStack/cluster-stacks-demo) repository. Although this is based on the not-production-ready Docker provider, the usage is the same for every provider.
+With the Cluster Stacks, in the V2 KaaS reference implementation, we provide an opinionated optimized configuration of Kubernetes cluster management based on [Kubernetes Cluster-API](https://cluster-api.sigs.k8s.io/).
+Through better packaging, integrated testing, and bundled configuration, SCS-based Kubernetes clusters provide easier individualization.
+Throughout the R6 development cycle Cluster Stacks were taken from a technical preview to be [functional and available on top of the IaaS reference implementation](https://github.com/SovereignCloudStack/issues/milestone/8) as well to replace the V1 KaaS reference implementation [k8s-cluster-api-provider](https://github.com/SovereignCloudStack/k8s-cluster-api-provider/).
+The Cluster Stacks have fully replaced V1 meanwhile to be the production-grade KaaS solution in SCS, please check out the
+[Quick Start Guide](/docs/container/components/cluster-stacks/providers/openstack/quickstart).
+For demo, test and development purposes, you can also try the [demo](https://github.com/SovereignCloudStack/cluster-stacks-demo) repository which is an implementation using the (not-for-production) Docker provider. Implementations for other infrastructure are intended; the one for Hetzner for example is maintained by syself itself.
 
 ### Public SCS Clouds in production