restructure and add info for self-service

Signed-off-by: Matthias Büchse <[email protected]>

Author: mbuechse

standards/certification/pipeline.md

@@ -1,9 +1,19 @@
 # SCS Compliance Check Pipeline Manual
 
-## How to add a new test subject in 8 easy steps
+The SCS compliance check suite runs automated tests, generates a signed report for the run, and feeds it to
+the compliance monitor. Roughly speaking, this process has to be performed daily, for instance, using a
+continuous-integration "pipeline".
 
-This is a how-to guide for adding your own subject if you so desire. This way will probably be quickest,
-but we are of course happy to help.
+Providers of public clouds do not need to use their own pipelines; those clouds can be tested via the
+official SCS compliance check pipeline.
+
+Alternatively, if using this pipeline is not feasible (for instance, for private clouds) or not desired,
+cloud-service providers can run the tests and feed the compliance monitor themselves.
+
+The next subsection shows common requirements for each of these two cases. The two subsections after that
+are each dedicated to the specific cases.
+
+## Common requirements for the compliance checks
 
 ### for SCS-compatible IaaS
 
@@ -14,17 +24,39 @@ to be used for purposes other than compliance testing (such as the
 
 1. Create an application credential. It must be possible to create resources such as servers, routers, etc.
 
-   In the next steps, we are going to create a pull request that is very similar to real-life example that
-   [adds the test subject `scaleup-occ2`](https://github.com/SovereignCloudStack/standards/pull/797).
-   (However, note that this example also adds an optional GitHub workflow, which we won't do here.)
-
 2. Create a new branch in [the standards repository](https://github.com/SovereignCloudStack/standards):
 
     - `git clone [email protected]:SovereignCloudStack/standards.git`
     - `cd standards`
     - `git checkout -b feat/add_my_cloud`
 
-3. Modify [playbooks/clouds.yaml.j2](https://github.com/SovereignCloudStack/standards/blob/main/playbooks/clouds.yaml.j2).
+3. Add your subject to the results table. This is necessary so your subject shows up in the
+   [compliance monitor web-site](https://compliance.sovereignit.cloud/page/table). Add the following lines
+   (substituting all-caps parts):
+
+   ```diff
+    | [gx-scs](https://github.com/SovereignCloudStack/docs/blob/main/community/cloud-resources/plusserver-gx-scs.md) | Dev environment provided for SCS & GAIA-X context | plusserver GmbH |
+    {#- #} [{{ results | pick('gx-scs', iaas) | summary }}]({{ detail_url('gx-scs', iaas) }}) {# -#}
+    | [HM](https://health.gx-scs.sovereignit.cloud:3000/) |
+   +| [SUBJECT_NAME](YOUR_URL) | DESCRIPTION | COMPANY_NAME |
+   +{#- #} [{{ results | pick('SUBJECT_NAME', iaas) | summary }}]({{ detail_url('SUBJECT_NAME', iaas) }}) {# -#}
+   +| [HM](HEALTH_MONITOR_URL) |
+   ```
+
+   Ideally, insert the lines at a position that keeps the part of the table below `gx-scs` sorted.
+
+Now you can proceed with the steps from one of the next subsections, depending on whether you want to
+check suite run by the SCS pipeline or not.
+
+## How to add a new test subject to the official pipeline
+
+### for SCS-compatible IaaS
+
+We are going to create a pull request that is very similar to real-life example that
+[adds the test subject `scaleup-occ2`](https://github.com/SovereignCloudStack/standards/pull/797).
+(However, note that this example also adds an optional GitHub workflow, which we won't do here.)
+
+1. Modify [playbooks/clouds.yaml.j2](https://github.com/SovereignCloudStack/standards/blob/main/playbooks/clouds.yaml.j2).
    This is necessary so that the tests can access your cloud.
    You can use the following template (replace all-caps parts):
 
@@ -42,7 +74,7 @@ to be used for purposes other than compliance testing (such as the
 
    Note that you need to replace dashes (and other special characters) by underscores in the last two lines.
 
-4. Add your subject to [Tests/config.toml](https://github.com/SovereignCloudStack/standards/blob/main/Tests/config.toml).
+2. Add your subject to [Tests/config.toml](https://github.com/SovereignCloudStack/standards/blob/main/Tests/config.toml).
    This is necessary so that your cloud will be included in the nightly tests. Add a line like so:
 
    ```diff
@@ -55,9 +87,9 @@ to be used for purposes other than compliance testing (such as the
    +    "SUBJECT_NAME",
    ```
 
-   Ideally insert your subject so that the list (after `gx-scs`) remains sorted.
+   Ideally, insert your subject so that the list (after `gx-scs`) remains sorted.
 
-5. Add your subject to [compliance-monitor/bootstrap.yaml](https://github.com/SovereignCloudStack/standards/blob/main/compliance-monitor/bootstrap.yaml).
+3. Add your subject to [compliance-monitor/bootstrap.yaml](https://github.com/SovereignCloudStack/standards/blob/main/compliance-monitor/bootstrap.yaml).
    This is necessary to that the reports will be accepted as genuine. Add a section like so:
 
    ```diff
@@ -71,22 +103,7 @@ to be used for purposes other than compliance testing (such as the
 
    Again, insert your subject so that the list (after `gx-scs`) remains sorted.
 
-6. Add your subject to the results table. This is necessary so your subject shows up in the
-   [compliance monitor web-site](https://compliance.sovereignit.cloud/page/table). Add the folling lines
-   (again substituting all-caps parts):
-
-   ```diff
-    | [gx-scs](https://github.com/SovereignCloudStack/docs/blob/main/community/cloud-resources/plusserver-gx-scs.md) | Dev environment provided for SCS & GAIA-X context | plusserver GmbH | 
-    {#- #} [{{ results | pick('gx-scs', iaas) | summary }}]({{ detail_url('gx-scs', iaas) }}) {# -#}
-    | [HM](https://health.gx-scs.sovereignit.cloud:3000/) |
-   +| [SUBJECT_NAME](YOUR_URL) | DESCRIPTION | COMPANY_NAME | 
-   +{#- #} [{{ results | pick('SUBJECT_NAME', iaas) | summary }}]({{ detail_url('SUBJECT_NAME', iaas) }}) {# -#}
-   +| [HM](HEALTH_MONITOR_URL) |
-   ```
-
-   Again, insert the lines at a position that keeps the table sorted (below `gx-scs`).
-
-7. Finally, add secrets to [.zuul.d/secure.yaml](https://github.com/SovereignCloudStack/standards/blob/main/.zuul.d/secure.yaml).
+4. Finally, add secrets to [.zuul.d/secure.yaml](https://github.com/SovereignCloudStack/standards/blob/main/.zuul.d/secure.yaml).
    This is necessary so the tests can access your cloud.
 
    This step is the most involved, and you can always have us do it for you; in that case, please send us
@@ -136,7 +153,64 @@ to be used for purposes other than compliance testing (such as the
 
    Note that you have to use the same keys as in Step 3 (that is, with special characters replaced).
 
-8. Commit your changes and open a pull request:
+5. Commit your changes and open a pull request:
+
+   ```shell
+   git commit -asm "Add SUBJECT_NAME"
+   git push  # the output of this command will show you the URL for creating the pull request
+   ```
+
+## How to feed the compliance monitor yourself
+
+Note: you may have to adapt these instructions to your infrastructure. For instance, the secrets
+we create here are stored locally. If you want to include the check suite into your own
+continuous-integration pipeline, you may want to use some dedicated credential store and mechanism for
+injecting secrets. You may want to take inspiration from our own Zuul setup by looking at
+[.zuul.d](https://github.com/SovereignCloudStack/standards/tree/main/.zuul.d) and
+[playbooks](https://github.com/SovereignCloudStack/standards/tree/main/playbooks).
+
+1. Install requirements.
+
+   ```shell
+   virtualenv .venv
+   . .venv/bin/activate
+   pip install -r requirements.txt
+   pip install passlib argon2_cffi  # these are only needed for step 2
+   ```
+
+2. Run our support script (substitute `$SUBJECT_NAME` appropriately):
+
+   ```shell
+   Tests/add_subject.py $SUBJECT_NAME
+   ```
+
+   Follow the instructions.
+   At the end, it will tell you to amend the file `compliance-monitor/boostrap.py`. Do so.
+
+3. Make sure you have `$SUBJECT_NAME` in your `.config/openstack/clouds.yaml` like so:
+
+   ```yaml
+     SUBJECT_NAME:
+       region_name: REGION
+       interface: "public"
+       identity_api_version: 3
+       auth_type: "v3applicationcredential"
+       auth:
+         auth_url: AUTH_URL
+         application_credential_id: "APPLICATION_CREDENTIAL_ID"
+         application_credential_secret: "APPLICATION_CREDENTIAL_SECRET"
+   ```
+
+4. Perform a test run of the check suite like so:
+
+   ```shell
+   python3 Tests/scs-test-runner.py --config Tests/config.toml run --scope scs-compatible-iaas --subject $SUBJECT_NAME
+   ```
+
+   If the report can't be submitted at the very end, this is to be expected, because the keyfile is not yet
+   known to the compliance monitor. To change this, we proceed to the next step.
+
+5. Commit your changes and open a pull request:
 
    ```shell
    git commit -asm "Add SUBJECT_NAME"