ci: rollout several recent changes to CI testing

* Pass in a YAML true value as `__bootc_validation: true` using
the --extra-vars option to ensure that `__bootc_validation` is
treated as a boolean and not a string value.

`-e "__bootc_validation: true"`

You can also use JSON format:

`-e '{"__bootc_validation": true}'`

but YAML is simpler in this case.

* Use tox-lsr version 3.11.1

* Ensure the citest bad comment works when the test was cancelled in
addition to the failure case.

* Update contributing.md documentation

* Update number of nodes to use in testing farm, if needed

* remove unnecessary ansible-lint skips

Signed-off-by: Rich Megginson <[email protected]>

Author: richm

.github/workflows/ansible-lint.yml

@@ -32,7 +32,7 @@ jobs:
       - name: Install tox, tox-lsr
         run: |
           set -euxo pipefail
-          pip3 install "git+https://github.com/linux-system-roles/[email protected].0"
+          pip3 install "git+https://github.com/linux-system-roles/[email protected].1"
 
       - name: Convert role to collection format
         id: collection

.github/workflows/ansible-managed-var-comment.yml

@@ -30,7 +30,7 @@ jobs:
       - name: Install tox, tox-lsr
         run: |
           set -euxo pipefail
-          pip3 install "git+https://github.com/linux-system-roles/[email protected].0"
+          pip3 install "git+https://github.com/linux-system-roles/[email protected].1"
 
       - name: Run ansible-plugin-scan
         run: |

.github/workflows/ansible-test.yml

@@ -33,7 +33,7 @@ jobs:
       - name: Install tox, tox-lsr
         run: |
           set -euxo pipefail
-          pip3 install "git+https://github.com/linux-system-roles/[email protected].0"
+          pip3 install "git+https://github.com/linux-system-roles/[email protected].1"
 
       - name: Convert role to collection format
         run: |

.github/workflows/qemu-kvm-integration-tests.yml

@@ -105,7 +105,7 @@ jobs:
           python3 -m pip install --upgrade pip
           sudo apt update
           sudo apt install -y --no-install-recommends git ansible-core genisoimage qemu-system-x86
-          pip3 install "git+https://github.com/linux-system-roles/[email protected].0"
+          pip3 install "git+https://github.com/linux-system-roles/[email protected].1"
 
       # HACK: Drop this when moving this workflow to 26.04 LTS
       - name: Update podman to 5.x for compatibility with bootc-image-builder's podman 5
@@ -199,7 +199,7 @@ jobs:
               if tox -e "$env" -- --image-file "$(pwd)/$image_file" \
                      --log-level debug $TOX_ARGS \
                      --lsr-report-errors-url DEFAULT \
-                     -e __bootc_validation=true \
+                     -e "__bootc_validation: true" \
                      -- "$test" >out 2>&1; then
                   mv out "${test}-PASS.log"
               else

.github/workflows/tft_citest_bad.yml

@@ -33,8 +33,9 @@ jobs:
             echo "The workflow $PENDING_RUN is still running, wait for it to finish to re-run"
             exit 1
           fi
+          # TF tests can fail or can be cancelled due to TF internal issues
           RUN_ID=$(gh api "repos/$REPO/actions/workflows/tft.yml/runs?event=issue_comment" \
-            | jq -r "[.workflow_runs[] | select( .display_title == \"$PR_TITLE\" ) | select( .conclusion == \"failure\" ) | .id][0]")
+            | jq -r "[.workflow_runs[] | select( .display_title == \"$PR_TITLE\" ) | select( .conclusion == \"failure\" or .conclusion == \"cancelled\" ) | .id][0]")
           if [ "$RUN_ID" = "null" ]; then
             echo "Failed workflow not found, exiting"
             exit 1

contributing.md

@@ -74,3 +74,49 @@ To run the tests locally in container, you can use the attached Containerfile
 podman build -t system-roles-ssh .dev-tools
 podman run -itv ./:/src:Z system-roles-ssh
 ```
+## Running CI Tests Locally
+
+### Use tox-lsr with qemu
+
+The latest version of tox-lsr supports qemu testing.
+<https://github.com/linux-system-roles/tox-lsr#qemu-testing>
+
+**Steps:**
+
+1. If you are using RHEL or CentOS, enable the EPEL repository for your
+   platform - <https://docs.fedoraproject.org/en-US/epel/>
+
+2. Use yum or dnf to install `standard-test-roles-inventory-qemu`
+   * If for some reason dnf/yum do not work, just download the script from
+     <https://pagure.io/standard-test-roles/raw/master/f/inventory/standard-inventory-qcow2> <!--- wokeignore:rule=master -->
+     * copy to your `$PATH`, and make sure it is executable
+
+3. Install tox
+   * Use yum/dnf to install `python3-tox` - if that does not work, then use
+     `pip install --user tox`, then make sure `~/.local/bin` is in your `$PATH`
+
+4. Install tox-lsr <https://github.com/linux-system-roles/tox-lsr#how-to-get-it>
+
+   ```bash
+   pip install --user git+https://github.com/linux-system-roles/tox-lsr@main
+   ```
+
+5. Download the config file to `~/.config/linux-system-roles.json` from
+   <https://github.com/linux-system-roles/linux-system-roles.github.io/blob/main/download/linux-system-roles.json>
+
+6. Assuming you are in a git clone of a role repo which has a tox.ini file -
+   you can use e.g.
+
+   ```bash
+   tox -e qemu-ansible-core-2.14 -- --image-name centos-9 tests/tests_default.yml
+   ```
+
+There are many command line options and environment variables which can be used
+to control the behavior, and you can customize the testenv in tox.ini. See
+<https://github.com/linux-system-roles/tox-lsr#qemu-testing>
+
+This method supports RHEL also - will download the latest image for a compose,
+and will set up the yum repos to point to internal composes.
+
+See <https://linux-system-roles.github.io/contribute.html> for general
+development guidelines.