Merge branch 'main' into feat/cluster-hardening-tests

Author: cah-patrickthiem

Standards/scs-0125-v1-secure-connections.md

@@ -84,4 +84,3 @@ If the standard is used by a provider, the following decisions are binding and v
 [k8s-ha]: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/
 [k8s-large-clusters]: https://kubernetes.io/docs/setup/best-practices/cluster-large/
 [scs-0213-v1]: https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0213-v1-k8s-nodes-anti-affinity.md
-[k8s-labels-docs]: https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesiozone

Standards/scs-0214-v1-k8s-node-distribution.md

@@ -1,7 +1,8 @@
 ---
 title: Kubernetes Node Distribution and Availability
 type: Standard
-status: Draft
+status: Stable
+stabilized_at: 2024-11-21
 replaces: scs-0214-v1-k8s-node-distribution.md
 track: KaaS
 ---
@@ -100,23 +101,6 @@ These labels MUST be kept up to date with the current state of the deployment.
   The field gets autopopulated most of the time by either the kubelet or external mechanisms
   like the cloud controller.
 
-- `topology.scs.community/host-id`
-
-  This is an SCS-specific label; it MUST contain the hostID of the physical machine running
-  the hypervisor (NOT: the hostID of a virtual machine). Here, the hostID is an arbitrary identifier,
-  which need not contain the actual hostname, but it should nonetheless be unique to the host.
-  This helps identify the distribution over underlying physical machines,
-  which would be masked if VM hostIDs were used.
-
-## Conformance Tests
-
-The script `k8s-node-distribution-check.py` checks the nodes available with a user-provided
-kubeconfig file. Based on the labels `topology.scs.community/host-id`,
-`topology.kubernetes.io/zone`, `topology.kubernetes.io/region` and `node-role.kubernetes.io/control-plane`,
-the script then determines whether the nodes are distributed according to this standard.
-If this isn't the case, the script produces an error.
-It also produces warnings and informational outputs, e.g., if labels don't seem to be set.
-
 ## Previous standard versions
 
 This is version 2 of the standard; it extends [version 1](scs-0214-v1-k8s-node-distribution.md) with the

Standards/scs-0214-v2-k8s-node-distribution.md

@@ -16,25 +16,15 @@ Worker nodes can also be distributed over "failure zones", but this isn't a requ
 Distribution must be shown through labelling, so that users can access these information.
 
 Node distribution metadata is provided through the usage of the labels
-`topology.kubernetes.io/region`, `topology.kubernetes.io/zone` and
-`topology.scs.community/host-id` respectively.
-
-At the moment, not all labels are set automatically by most K8s cluster utilities, which incurs
-additional setup and maintenance costs.
+`topology.kubernetes.io/region` and `topology.kubernetes.io/zone`.
 
 ## Automated tests
 
-### Notes
-
-The test for the [SCS K8s Node Distribution and Availability](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0214-v2-k8s-node-distribution.md)
-checks if control-plane nodes are distributed over different failure zones (distributed into
-physical machines, zones and regions) by observing their labels defined by the standard.
-
-### Implementation
+Currently, automated testing is not readily possible because we cannot access information about
+the underlying host of a node (as opposed to its region and zone). Therefore, the test will only output
+a tentative result.
 
-The script [`k8s_node_distribution_check.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/kaas/k8s-node-distribution/k8s_node_distribution_check.py)
-connects to an existing K8s cluster and checks if a distribution can be detected with the labels
-set for the nodes of this cluster.
+The current implementation can be found in the script [`k8s_node_distribution_check.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/kaas/k8s-node-distribution/k8s_node_distribution_check.py).
 
 ## Manual tests
 

Standards/scs-0214-w1-k8s-node-distribution-implementation-testing.md

@@ -1,2 +1,3 @@
 htmlcov/
 .coverage
+.secret

Tests/.gitignore

@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+# vim: set ts=4 sw=4 et:
+#
+# add_subject.py
+#
+# (c) Matthias Büchse <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+import base64
+import getpass
+import os
+import os.path
+import re
+import shutil
+import signal
+import subprocess
+import sys
+
+try:
+    from passlib.context import CryptContext
+    import argon2  # noqa:F401
+except ImportError:
+    print('Missing passlib and/or argon2. Please do:\npip install passlib argon2_cffi', file=sys.stderr)
+    sys.exit(1)
+
+# see ../compliance-monitor/monitor.py
+CRYPTCTX = CryptContext(schemes=('argon2', 'bcrypt'), deprecated='auto')
+SSH_KEYGEN = shutil.which('ssh-keygen')
+SUBJECT_RE = re.compile(r"[a-zA-Z0-9_\-]+")
+
+
+def main(argv, cwd):
+    if len(argv) != 1:
+        raise RuntimeError("Need to supply precisely one argument: name of subject")
+    subject = argv[0]
+    print(f"Attempt to add subject {subject!r}")
+    keyfile_path = os.path.join(cwd, '.secret', 'keyfile')
+    tokenfile_path = os.path.join(cwd, '.secret', 'tokenfile')
+    if os.path.exists(keyfile_path):
+        raise RuntimeError(f"Keyfile {keyfile_path} already present. Please proceed manually")
+    if os.path.exists(tokenfile_path):
+        raise RuntimeError(f"Tokenfile {tokenfile_path} already present. Please proceed manually")
+    if not SUBJECT_RE.fullmatch(subject):
+        raise RuntimeError(f"Subject name {subject!r} using disallowed characters")
+    sanitized_subject = subject.replace('-', '_')
+    print("Creating API key...")
+    while True:
+        password = getpass.getpass("Enter passphrase: ")
+        if password == getpass.getpass("Repeat passphrase: "):
+            break
+        print("No match. Try again...")
+    token = base64.b64encode(f"{subject}:{password}".encode('utf-8'))
+    hash_ = CRYPTCTX.hash(password)
+    with open(tokenfile_path, "wb") as fileobj:
+        os.fchmod(fileobj.fileno(), 0o600)
+        fileobj.write(token)
+    print("Creating key file using `ssh-keygen`...")
+    subprocess.check_call([SSH_KEYGEN, '-t', 'ed25519', '-C', sanitized_subject, '-f', keyfile_path, '-N', '', '-q'])
+    with open(keyfile_path + '.pub', "r") as fileobj:
+        pubkey_components = fileobj.readline().split()
+    print(f'''
+The following SECRET files have been created:
+
+  - {keyfile_path}
+  - {tokenfile_path}
+
+They are required for submitting test reports. You MUST keep them secure and safe.
+
+Insert the following snippet into compliance-monitor/bootstrap.yaml:
+
+  - subject: {subject}
+    api_keys:
+      - "{hash_}"
+    keys:
+      - public_key: "{pubkey_components[1]}"
+        public_key_type: "{pubkey_components[0]}"
+        public_key_name: "primary"
+
+Make sure to submit a pull request with the changed file. Otherwise, the reports cannot be submitted.
+''', end='')
+
+
+if __name__ == "__main__":
+    try:
+        sys.exit(main(sys.argv[1:], cwd=os.path.dirname(sys.argv[0]) or os.getcwd()) or 0)
+    except RuntimeError as e:
+        print(str(e), file=sys.stderr)
+        sys.exit(1)
+    except KeyboardInterrupt:
+        print("Interrupted", file=sys.stderr)
+        sys.exit(128 + signal.SIGINT)

Tests/add_subject.py

@@ -0,0 +1,61 @@
+# Secure Connections Standard Test Suite
+
+## Test Environment Setup
+
+> **NOTE:** The test execution procedure does not require cloud admin rights.
+
+A valid cloud configuration for the OpenStack SDK in the shape of "`clouds.yaml`" is mandatory[^1].
+**This file is expected to be located in the current working directory where the test script is executed unless configured otherwise.**
+
+[^1]: [OpenStack Documentation: Configuring OpenStack SDK Applications](https://docs.openstack.org/openstacksdk/latest/user/config/configuration.html)
+
+The test execution environment can be located on any system outside of the cloud infrastructure that has OpenStack API access.
+Make sure that the API access is configured properly in "`clouds.yaml`".
+
+It is recommended to use a Python virtual environment[^2].
+Next, install the libraries required by the test suite:
+
+```bash
+pip3 install openstacksdk sslyze
+```
+
+> Note: the version of the sslyze library determines the [version of the Mozilla TLS recommendation JSON](https://wiki.mozilla.org/Security/Server_Side_TLS#JSON_version_of_the_recommendations) that it checks against.
+
+Within this environment execute the test suite.
+
+[^2]: [Python 3 Documentation: Virtual Environments and Packages](https://docs.python.org/3/tutorial/venv.html)
+
+## Test Execution
+
+The test suite is executed as follows:
+
+```bash
+python3 tls-checker.py --os-cloud mycloud
+```
+
+As an alternative to "`--os-cloud`", the "`OS_CLOUD`" environment variable may be specified instead.
+The parameter is used to look up the correct cloud configuration in "`clouds.yaml`".
+For the example command above, this file should contain a `clouds.mycloud` section like this:
+
+```yaml
+---
+clouds:
+  mycloud:
+    auth:
+      auth_url: ...
+      ...
+    ...
+```
+
+For any further options consult the output of "`python3 tls-checker.py --help`".
+
+### Script Behavior & Test Results
+
+The script will print all actions and passed tests to `stdout`.
+
+If all tests pass, the script will return with an exit code of `0`.
+
+If any test fails, the script will halt, print the exact error to `stderr` and return with a non-zero exit code.
+
+Any tests that indicate a recommendation of the standard is not met, will print a warning message under the corresponding endpoint output.
+However, unmet recommendations will not count as errors.