Merge branch 'add_larger_flavors' of github.com:/SovereignCloudStack/standards into add_larger_flavors

Author: garloff

.zuul.d/secure.yaml

@@ -79,6 +79,28 @@
           o9slrGAyL7g3zdjNlUJA04U33SNCvaCxL8fac6JZ15vrqeW4g4AB4+rx7fYKAnOVg2FOL
           5jMOKsiGgfLvz1KZ9c6Q1ThfeCQzG9waWJnyCx2R2tEtyQ17hIW6Rzo1RzmQkUyvLN9TJ
           CLSZCUoR+2Ut+ZlpDi3vVushWWLXyjj8ojblTO/zqlbQ1A+7d/C+5x2mrh2T/0=
+      cnds_ro_ac_id: !encrypted/pkcs1-oaep
+        - l2IMEJwkw8xqgzKvCkPgGVzoBNiMrj/+oNfq3dvFyLU8f/AJ8XDMsmaeNBj/hSGY6O3ar
+          Qs8ckn6BUTD253Ft8izsvv7E535KN/o5IhN7f+juKri1jVus/XLkrx3t3exHL1piSy0y/
+          y5k1FGpEclmzyEdtaCorEOQraXRCLAOmyYba6aCt5YhPVJkOjv8Aupy7Y/tSHXdsFKgZt
+          fJItALWehZVbYtl4WHpmrPwV0uW8mKo/T41o2aJDJ3a2BRodqVvTNSZb9YNnLyBkxW+Kc
+          w1AAk9E2U+tinWxFAJQAE42JZIesv6F9SoJhl9ViYsENjNtwdpndrrF1j2BmmiqJ1kVwp
+          y4UHnheNUBsIXe2RUnRq2z0m0rQ/kyQQSTluUV0QGnb34a3GuMqPCsiAAbFRuL6Ax8zXu
+          UoQ+C6BCNXJJzyjbJC9CLILqHLqZUCkYimiBf4+GmoDEANNi5FUZgtwK0p9TJN/7KvLJ0
+          h+73PtoCnVrnsYcaEu+tJO3Jfm43tilTRixTtVbWL1F+dgnBCdh3dFHm3l16npEMxpyR9
+          2P6BKyeROBAwaBURc3UhtqZjwMc+YmYLGXRDjd/DKyLHJ0j27ONWCtQHbRzJZjxfvrMfC
+          dyowl8wOxpgd3EiUDDufncD7JmKBcyJRQOvTTGJz6T3cP8h9b65103faoyVRo0=
+      cnds_ro_ac_secret: !encrypted/pkcs1-oaep
+        - KvlXdw4RIkzHwaZPeeI+lhycJboSRBh7DJ/43Q0sNP9o1vkJG72pSv8w2HhycZU95SU6g
+          k0B5uMwIHpXiQnmjgA4f8lLMkI6ppA5FLL5F3LwsVIlfUI1x8aM8Zl+LyVHSZJ/0kP3l7
+          QTEJ6DDNVdI6xftCkKQiABUCMYwgbZsU3c3rJeF/RAuCUtrs/gRv+2F4/es7UWaafYvQa
+          gqx4nC7LGn+7q1UH4BIbwVQdjH8f7H1SSEkz4t1goNqgVMqqv24hNF/KMRRGfZ/Zv9zPt
+          B+uczjb4Jc6zwJL/zF+sZc1pIt9zn/RijJTYv0BX+ldfMiOflST/FlXcMZqULXHnyQLK/
+          iqaJcTfI7fv1OCUtpNc0n6dJhK8piU//1JQ3Yanov0QTdEo4OTRxirGxIobzJfFl+hf+8
+          D5b1ZKqkPhoTGP/vjl1XzvV2QuJ+ZX6P9GWKJr4r/9b2RuwywD71fUbXqmEva3/THY2Sg
+          gY6QHocYpATL46iLkv97QANNUxTdxL7hQjdl/tf3TAHjCclmxdWhBJdvCJN/1xCM6EgVp
+          NykBYxJ+kxSmkcFCSdUM8Td75bA/UzkPCdix1reJMdEAxTE9fC55XQ/liTLlGquQDnZty
+          VLDH7x3ZJcxZsvqKR6vNbYYzJvDPTBYpHrhD7kx3ubyO9KX+SzZ+Dfhe9M8T8U=
       poc_wgcloud_ac_id: !encrypted/pkcs1-oaep
         - dQIs3NJt1CpP1925+b9QjjwonqjmiuCl1ewxw160yIEHQ/qyQiwutJbsg4IYS9XKhKc2X
           GumOOpLY7+/uNRR5pZmEfOdlGnPoJvVhYtCqHBFy7xQ6NLHKFxCT8zHM9ppSl1Hjc2G2F

Standards/scs-0103-v1-standard-flavors.md

@@ -7,7 +7,7 @@ track: IaaS
 description: |
   The SCS-0103 standard outlines mandatory and recommended specifications for flavors and properties in OpenStack
   environments to ensure uniformity across SCS clouds. Mandatory and recommended flavors are defined with specific
-  configurations of vCPUs, vCPU types, RAM, and root disk sizes, alongside extra specs like scs:name-vN, scs:cpu-type,
+  configurations of vCPUs, vCPU types, RAM, and root disk sizes, alongside extra_specs like scs:name-vN, scs:cpu-type,
   and scs:diskN-type to detail the flavor's specifications. This standard facilitates guaranteed availability and
   consistency of flavors, simplifying the deployment process for DevOps teams.
 ---
@@ -18,7 +18,7 @@ Note that this is v1.2 of this standard. See the closing section for more detail
 
 ## Terminology
 
-extra_specs
+extra\_specs:
   Additional properties on an OpenStack flavor, see
   [OpenStack Nova user documentation](https://docs.openstack.org/nova/2024.1/user/flavors.html#extra-specs)
   and
@@ -33,9 +33,9 @@ OpenStack providers thus typically offer a large selection of flavors.
 While flavors can be discovered (`openstack flavor list`), it is helpful for users (DevOps teams),
 to have a guaranteed set of flavors available on all SCS clouds, so these need not be discovered.
 
-## Properties (extra_specs)
+## Properties (extra\_specs)
 
-The following extra_specs are recognized, together with the respective semantics:
+The following extra\_specs are recognized, together with the respective semantics:
 
 - `scs:name-vN=NAME` (where `N` is a positive integer, and `NAME` is some string) means that
   `NAME` is a valid name for this flavor according to any major version of the [SCS standard on
@@ -53,22 +53,22 @@ The following extra_specs are recognized, together with the respective semantics
 
 Whenever ANY of these are present on ANY flavor, the corresponding semantics must be satisfied.
 
-The extra_spec `scs:name-vN` is to be interpreted as "name variant N". This name scheme is designed to be
+The extra\_spec `scs:name-vN` is to be interpreted as "name variant N". This name scheme is designed to be
 backwards compatible with v1.0 of this standard, where `scs:name-vN` is interpreted as
 "name according to naming standard vN". We abandon this former interpretation for two reasons:
 
 1. the naming standards admit multiple (even many) names for the same flavor, and we want to provide a means
    of advertising more than one of them (said standards recommend using two: a short one and a long one),
 2. the same flavor name may be valid according to multiple versions at the same time, which would lead to
-   a pollution of the extra_specs with redundant properties; for instance, the name
+   a pollution of the extra\_specs with redundant properties; for instance, the name
    `SCS-4V-16` is valid for both [scs-0100-v2](scs-0100-v2-flavor-naming.md) and
    [scs-0100-v3](scs-0100-v3-flavor-naming.md), and, since it does not use any extension, it will be valid
    for any future version that only changes the extensions, such as the GPU vendor and architecture.
 
 Note that it is not required to use consecutive numbers to number the name variants.
 This way, it becomes easier to remove a single variant (no "closing the gap" required).
 
-If extra_specs of the form `scs:name-vN` are used to specify SCS flavor names, it is RECOMMENDED to include
+If extra\_specs of the form `scs:name-vN` are used to specify SCS flavor names, it is RECOMMENDED to include
 names for the latest stable major version of the standard on flavor naming.
 
 ## Standard SCS flavors
@@ -122,17 +122,17 @@ flavors with more RAM than the ones above, they should try to use these.
 | Recommended name | vCPUs  | vCPU type     | RAM [GiB]  | Root disk [GB]  | Disk type  |
 | ---------------- | ------ | ------------- | ---------- | --------------- | ---------- |
 | SCS-16V-64       |     16 | shared-core   |         64 |                 |            |
-| SCS-16V-64-100   |     16 | shared-core   |         64 |             100 | (any)      |
 | SCS-8V-64        |      8 | shared-core   |         64 |                 |            |
 | SCS-16V-128      |     16 | shared-core   |        128 |                 |            |
-| SCS-8V-64-100    |      8 | shared-core   |         64 |             100 | (any)      |
-| SCS-16V-128-100  |     16 | shared-core   |        128 |             100 | (any)      |
-| SCS-4V-64        |      4 | shared-core   |         64 |                 |            |
-| SCS-8V-128       |      8 | shared-core   |        128 |                 |            |
-| SCS-4V-64-100    |      4 | shared-core   |         64 |             100 | (any)      |
-| SCS-8V-128-100   |      8 | shared-core   |        128 |             100 | (any)      |
-| SCS-4V-128       |      4 | shared-core   |        128 |                 |            |
-| SCS-4V-128-100   |      4 | shared-core   |        128 |             100 | (any)      |
+
+Note that no flavors with disks have been added here; providers are of course welcome to
+also add variants with unspecified (e.g. `-200`) or ssd+ (e.g. `-200s`) disk types.
+Sticking to the 5, 10, 20, 50, 100, 200, 500, 1000 schedule for disk sizes is recommended
+in that case to avoid unnecessary fragmentation.
+
+Likewise, flavors with more vCPUs (e.g. `-32V`, `-64V`) may be added and we recommend
+sticking to powers of two and to keep the vCPU to GiB RAM ratios 1:2, 1:4 and 1:8,
+unless customers have very specific demands.
 
 ### Guarantees and properties
 

Standards/scs-0103-w1-standard-flavors-implementation.md

@@ -9,7 +9,10 @@ supplements:
 ## Operational tooling
 
 The [openstack-flavor-manager](https://github.com/osism/openstack-flavor-manager) is able to
-create all standard, mandatory SCS flavors for you. It takes input that can be generated by
+create all standard, mandatory as well as recommended SCS flavors for you. It now has a `--limit-memory`
+(defaulting to 32 GiB) to skip the creation of recommended flavors above this memory limit.
+
+You can generate input for it using the tool
 [`flavor-manager-input.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/scs_0100_flavor_naming/flavor-manager-input.py).
 
 ## Automated tests

Tests/config.toml

@@ -19,6 +19,7 @@ scopes = [
 subjects = [
     "scs2",
     "artcodix",
+    "artcodix-ro",
     # currently not reachable from outside: "cc-rrze",
     "pco-prod1",
     "pco-prod2",

Tests/iaas/openstack_test.py

@@ -40,7 +40,7 @@
 from scs_0115_security_groups.security_groups import \
     compute_scs_0115_default_rules
 from scs_0116_key_manager.key_manager import \
-    compute_services_lookup, compute_scs_0116_presence, compute_scs_0116_permissions
+    ensure_unprivileged, compute_services_lookup, compute_scs_0116_presence, compute_scs_0116_permissions
 from scs_0117_volume_backup.volume_backup import \
     compute_scs_0117_test_backup
 from scs_0123_mandatory_services.mandatory_services import \
@@ -280,6 +280,20 @@ def harness(name, *check_fns):
     print(f"{name}: {result}")
 
 
+def run_sanity_checks(container):
+    # make sure that we can connect to the cloud and that the user doesn't have elevated privileges
+    # the former would lead to each testcase aborting with a marginally useful message;
+    # the latter would lead to scs_0116_permissions aborting, which we don't want to single out
+    try:
+        conn = container.conn
+    except openstack.exceptions.ConfigException:
+        logger.critical("Please make sure that ~/.config/openstack/clouds.yaml exists and is correct!")
+        raise
+    if "member" not in ensure_unprivileged(conn, quiet=True):
+        logger.critical("Please make sure that your OpenStack user has role member.")
+        raise RuntimeError("OpenStack user is missing member role.")
+
+
 def main(argv):
     # configure logging, disable verbose library logging
     logging.basicConfig(format='%(levelname)s: %(message)s', level=logging.DEBUG)
@@ -320,6 +334,7 @@ def main(argv):
         sys.exit(1)
 
     c = make_container(cloud)
+    run_sanity_checks(c)
     for testcase in testcases:
         testcase_name = testcase.rsplit('/', 1)[0]  # see the note above
         harness(testcase_name, lambda: getattr(c, testcase.replace('-', '_').replace('/', '_')))

Tests/iaas/scs_0116_key_manager/key_manager.py

@@ -7,7 +7,7 @@
 logger = logging.getLogger(__name__)
 
 
-def ensure_unprivileged(conn: openstack.connection.Connection) -> list:
+def ensure_unprivileged(conn: openstack.connection.Connection, quiet=False) -> list:
     """
     Retrieves role names.
     Raises exception if elevated privileges (admin, manager) are present.
@@ -19,6 +19,8 @@ def ensure_unprivileged(conn: openstack.connection.Connection) -> list:
     role_names = set(conn.session.auth.get_access(conn.session).role_names)
     if role_names & {"admin", "manager"}:
         raise RuntimeError("user privileges too high: admin/manager roles detected")
+    if quiet:
+        return role_names
     if "reader" in role_names:
         logger.info("User has reader role.")
     custom_roles = sorted(role_names - {"reader", "member"})

compliance-monitor/bootstrap.yaml

@@ -19,6 +19,11 @@ accounts:
     delegates:
       - zuul_ci
   - subject: artcodix
+    group: artcodix
+    delegates:
+      - zuul_ci
+  - subject: artcodix-ro
+    group: artcodix
     delegates:
       - zuul_ci
   - subject: cc-rrze

compliance-monitor/templates/overview.md.j2

@@ -19,8 +19,8 @@ Version numbers are suffixed by a symbol depending on state: * for _draft_, †
 | [CC@RRZE](https://www.rrze.fau.de/) | Private Compute Cloud (CC) for [FAU](https://www.fau.de/) | Regionales Rechenzentrum Erlangen | 
 {#- #} [{{ results | pick(iaas, 'cc-rrze') | summary }}]({{ detail_url('cc-rrze', iaas) }}) {# -#}
 | (soon) |
-| [CNDS](https://cnds.io/) | Public cloud for customers | artcodix GmbH |
-{#- #} [{{ results | pick(iaas, 'artcodix') | summary }}]({{ detail_url('artcodix', iaas) }}) {# -#}
+| [CNDS](https://cnds.io/) | Public cloud for customers (2 regions) | artcodix GmbH |
+{#- #} [{{ results | pick(iaas, 'artcodix', 'artcodix-ro') | summary }}]({{ detail_url('group-artcodix', iaas) }}) {# -#}
 | [HM](https://ohm.muc.cloud.cnds.io/) |
 | [pluscloud open](https://www.plusserver.com/en/products/pluscloud-open) | Public cloud for customers (4 regions)  | plusserver GmbH | {# #}
 {#- #}[{{ results | pick(iaas, 'pco-prod1', 'pco-prod2', 'pco-prod3', 'pco-prod4') | summary }}]({{ detail_url('group-pco-prod', iaas) }}) {# -#}

playbooks/clouds.yaml.j2

@@ -13,12 +13,21 @@ clouds:
     interface: public
     identity_api_verion: 3
     auth_type: "v3applicationcredential"
-    #region_name: "MUC"
+    region_name: "MUC"
     auth:
       auth_url: https://api.dc1.muc.cloud.cnds.io:5000/
       application_credential_id: "{{ clouds_conf.cnds_ac_id }}"
       application_credential_secret: "{{ clouds_conf.cnds_ac_secret }}"
       #project_id: 225a7363dab74b69aa1e3f744aced109
+  artcodix-ro:
+    interface: public
+    identity_api_verion: 3
+    auth_type: "v3applicationcredential"
+    region_name: "RO"
+    auth:
+      auth_url: https://api.dc1.ro.cloud.cnds.io:5000/
+      application_credential_id: "{{ clouds_conf.cnds_ro_ac_id }}"
+      application_credential_secret: "{{ clouds_conf.cnds_ro_ac_secret }}"
   cc-rrze:
     region_name: "DE-ERL"
     interface: "public"