Merge pull request #454 from validatedpatterns/main

sync: main to rhdp-deploy

Author: mbaldessari

common/.github/workflows/pattern-sh-ci.yml

@@ -20,10 +20,14 @@ jobs:
         # Fedora is not an option yet
         os: [ubuntu-latest, ubuntu-22.04]
     runs-on: ${{ matrix.os }}
+    permissions:
+      contents: read
 
     steps:
       - name: Checkout Repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v5
+        with:
+          persist-credentials: false
 
       - name: Install Podman on Ubuntu
         if: contains(matrix.os, 'ubuntu')

common/.github/workflows/superlinter.yml

@@ -9,19 +9,22 @@ jobs:
     name: Super linter
     # Set the agent to run on
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
 
     steps:
       - name: Checkout Code
-        uses: actions/checkout@v4
+        uses: actions/checkout@v5
         with:
           # Full git history is needed to get a proper list of changed files within `super-linter`
           fetch-depth: 0
+          persist-credentials: false
 
       ################################
       # Run Linter against code base #
       ################################
       - name: Lint Code Base
-        uses: super-linter/super-linter/slim@v7
+        uses: super-linter/super-linter/slim@ffde3b2b33b745cb612d787f669ef9442b1339a6 # v8.1.0
         env:
           VALIDATE_ALL_CODEBASE: true
           DEFAULT_BRANCH: main
@@ -33,12 +36,7 @@ jobs:
           VALIDATE_JSCPD: false
           VALIDATE_JSON_PRETTIER: false
           VALIDATE_MARKDOWN_PRETTIER: false
-          VALIDATE_KUBERNETES_KUBECONFORM: false
           VALIDATE_PYTHON_PYLINT: false
           VALIDATE_SHELL_SHFMT: false
           VALIDATE_YAML: false
           VALIDATE_YAML_PRETTIER: false
-          # VALIDATE_DOCKERFILE_HADOLINT: false
-          # VALIDATE_MARKDOWN: false
-          # VALIDATE_NATURAL_LANGUAGE: false
-          # VALIDATE_TEKTON: false

common/Changes.md

@@ -2,119 +2,119 @@
 
 ## Sep 24, 2024
 
-* Ansible has been moved out of the common code tree, you must use a clustergroup chart that is >= 0.9.1
+- Ansible has been moved out of the common code tree, you must use a clustergroup chart that is >= 0.9.1
 
 ## Sep 6, 2024
 
-* Most charts have been removed from the tree. To get the charts you now have to point to them
+- Most charts have been removed from the tree. To get the charts you now have to point to them
 
 ## Sep 25, 2023
 
-* Upgraded ESO to v0.9.5
+- Upgraded ESO to v0.9.5
 
 ## Aug 17, 2023
 
-* Introduced support for multisource applications via .chart + .chartVersion
+- Introduced support for multisource applications via .chart + .chartVersion
 
 ## Jul 8, 2023
 
-* Introduced a default of 20 for sync failures retries in argo applications (global override via global.options.applicationRetryLimit
+- Introduced a default of 20 for sync failures retries in argo applications (global override via global.options.applicationRetryLimit
   and per-app override via .syncPolicy)
 
 ## May 22, 2023
 
-* Upgraded ESO to 0.8.2
-* *Important* we now use the newly blessed sso config for argo. This means that gitops < 1.8 are *unsupported*
+- Upgraded ESO to 0.8.2
+- *Important* we now use the newly blessed sso config for argo. This means that gitops < 1.8 are *unsupported*
 
 ## May 18, 2023
 
-* Introduce a EXTRA_HELM_OPTS env variable that will be passed to the helm invocations
+- Introduce a EXTRA_HELM_OPTS env variable that will be passed to the helm invocations
 
 ## April 21, 2023
 
-* Added labels and annotation support to namespaces.yaml template
+- Added labels and annotation support to namespaces.yaml template
 
 ## Apr 11, 2023
 
-* Apply the ACM ocp-gitops-policy everywhere but the hub
+- Apply the ACM ocp-gitops-policy everywhere but the hub
 
 ## Apr 7, 2023
 
-* Moved to gitops-1.8 channel by default (stable is unmaintained and will be dropped starting with ocp-4.13)
+- Moved to gitops-1.8 channel by default (stable is unmaintained and will be dropped starting with ocp-4.13)
 
 ## March 20, 2023
 
-* Upgraded ESO to 0.8.1
+- Upgraded ESO to 0.8.1
 
 ## February 9, 2023
 
-* Add support for /values-<platform>.yaml and for /values-<platform>-<clusterversion>.yaml
+- Add support for /values-<platform>.yaml and for /values-<platform>-<clusterversion>.yaml
 
 ## January 29, 2023
 
-* Stop extracting the HUB's CA via an imperative job running on the imported cluster.
+- Stop extracting the HUB's CA via an imperative job running on the imported cluster.
   Just use ACM to push the HUB's CA out to the managed clusters.
 
 ## January 23, 2023
 
-* Add initial support for running ESO on ACM-imported clusters
+- Add initial support for running ESO on ACM-imported clusters
 
 ## January 18, 2023
 
-* Add validate-schema target
+- Add validate-schema target
 
 ## January 13, 2023
 
-* Simplify the secrets paths when using argo hosted sites
+- Simplify the secrets paths when using argo hosted sites
 
 ## January 10, 2023
 
-* vaultPrefixes is now optional in the v2 secret spec and defaults to ["hub"]
+- vaultPrefixes is now optional in the v2 secret spec and defaults to ["hub"]
 
 ## December 9, 2022
 
-* Dropped insecureUnsealVaultInsideCluster (and file_unseal) entirely. Now
+- Dropped insecureUnsealVaultInsideCluster (and file_unseal) entirely. Now
   vault is always unsealed via a cronjob in the cluster. It is recommended to
   store the imperative/vaultkeys secret offline securely and then delete it.
 
 ## December 8, 2022
 
-* Removed the legacy installation targets:
+- Removed the legacy installation targets:
   `deploy upgrade legacy-deploy legacy-upgrade`
   Patterns must now use the operator-based installation
 
 ## November 29, 2022
 
-* Upgraded vault-helm to 0.23.0
-* Enable vault-ssl by default
+- Upgraded vault-helm to 0.23.0
+- Enable vault-ssl by default
 
 ## November 22, 2022
 
-* Implemented a new format for the values-secret.yaml. Example can be found in examples/ folder
-* Now the order of values-secret file lookup is the following:
+- Implemented a new format for the values-secret.yaml. Example can be found in examples/ folder
+- Now the order of values-secret file lookup is the following:
   1. ~/values-secret-<patternname>.yaml
-  2. ~/values-secret.yaml
-  3. <patterngitrepo>/values-secret.yaml.template
-* Add support for ansible vault encrypted values-secret files. You can now encrypt your values-secret file
+  1. ~/values-secret.yaml
+  1. <patterngitrepo>/values-secret.yaml.template
+- Add support for Ansible vault encrypted values-secret files. You can now encrypt your values-secret file
   at rest with `ansible-vault encrypt ~/values-secret.yaml`. When running `make load-secrets` if an encrypted
   file is encountered the user will be prompted automatically for the password to decrypt it.
 
 ## November 6, 2022
 
-* Add support for /values-<CloudPlatform>-<clusterGroup>.yaml (e.g. /values-AWS-group-one.yaml)
+- Add support for /values-<CloudPlatform>-<clusterGroup>.yaml (e.g. /values-AWS-group-one.yaml)
 
 ## October 28, 2022
 
-* Updated vault helm chart to v0.22.1 and vault containers to 1.12.0
+- Updated vault helm chart to v0.22.1 and vault containers to 1.12.0
 
 ## October 25, 2022
 
-* Updated External Secrets Operator to v0.6.0
-* Moved to -UBI based ESO containers
+- Updated External Secrets Operator to v0.6.0
+- Moved to -UBI based ESO containers
 
 ## October 13, 2022
 
-* Added global.clusterVersion as a new helm variable which represents the OCP
+- Added global.clusterVersion as a new helm variable which represents the OCP
   Major.Minor cluster version. By default now a user can add a
   values-<ocpversion>-<clustergroup>.yaml file to have specific cluster version
   overrides (e.g. values-4.10-hub.yaml). Will need Validated Patterns Operator >= 0.0.6
@@ -123,20 +123,25 @@
 
 ## October 4, 2022
 
-* Extended the values-secret.yaml file to support multiple vault paths and re-wrote
+- Extended the values-secret.yaml file to support multiple vault paths and re-wrote
   the push_secrets feature as python module plugin. This requires the following line
   in a pattern's ansible.cfg's '[defaults]' stanza:
 
   `library=~/.ansible/plugins/modules:./ansible/plugins/modules:./common/ansible/plugins/modules:/usr/share/ansible/plugins/modules`
 
 ## October 3, 2022
 
-* Restore the ability to install a non-default site: `make TARGET_SITE=mysite install`
-* Revised tests (new output and filenames, requires adding new result files to Git)
-* ACM 2.6 required for ACM-based managed sites
-* Introduced global.clusterDomain template variable (without the `apps.` prefix)
-* Removed the ability to send specific charts to another cluster, use hosted argo sites instead
-* Added the ability to have the hub host `values-{site}.yaml` for spoke clusters.
+- Restore the ability to install a non-default site: `make TARGET_SITE=mysite install`
+
+- Revised tests (new output and filenames, requires adding new result files to Git)
+
+- ACM 2.6 required for ACM-based managed sites
+
+- Introduced global.clusterDomain template variable (without the `apps.` prefix)
+
+- Removed the ability to send specific charts to another cluster, use hosted argo sites instead
+
+- Added the ability to have the hub host `values-{site}.yaml` for spoke clusters.
 
   The following example would deploy the namespaces, subscriptions, and
   applications defined in `values-group-one.yaml` to the `perth` cluster

common/Makefile

@@ -7,7 +7,7 @@ endif
 # Set this to true if you want to skip any origin validation
 DISABLE_VALIDATE_ORIGIN ?= false
 ifeq ($(DISABLE_VALIDATE_ORIGIN),true)
-  VALIDATE_ORIGIN := 
+  VALIDATE_ORIGIN :=
 else
   VALIDATE_ORIGIN := validate-origin
 endif
@@ -31,9 +31,8 @@ TARGET_BRANCH ?= $(shell git rev-parse --abbrev-ref HEAD)
 #default to the branch remote
 TARGET_ORIGIN ?= $(shell git config branch.$(TARGET_BRANCH).remote)
 
-# This is to ensure that whether we start with a git@ or https:// URL, we end up with an https:// URL
-# This is because we expect to use tokens for repo authentication as opposed to SSH keys
-TARGET_REPO=$(shell git ls-remote --get-url --symref $(TARGET_ORIGIN) | sed -e 's/.*URL:[[:space:]]*//' -e 's%^git@%%' -e 's%^https://%%' -e 's%:%/%' -e 's%^%https://%')
+# The URL for the configured origin (could be HTTP/HTTPS/SSH)
+TARGET_REPO_RAW := $(shell git ls-remote --get-url --symref $(TARGET_ORIGIN))
 
 UUID_FILE ?= ~/.config/validated-patterns/pattern-uuid
 UUID_HELM_OPTS ?=
@@ -50,13 +49,22 @@ TOKEN_SECRET ?=
 TOKEN_NAMESPACE ?=
 
 ifeq ($(TOKEN_SECRET),)
-  HELM_OPTS=-f values-global.yaml --set main.git.repoURL="$(TARGET_REPO)" --set main.git.revision=$(TARGET_BRANCH) $(TARGET_SITE_OPT) $(UUID_HELM_OPTS) $(EXTRA_HELM_OPTS)
+  # SSH agents are not created for public repos (repos with no secret token) by the patterns operator so we convert to HTTPS
+  TARGET_REPO := $(shell echo "$(TARGET_REPO_RAW)" | sed 's/^git@\(.*\):\(.*\)/https:\/\/\1\/\2/')
+  SECRET_OPTS :=
 else
-  # When we are working with a private repository we do not escape the git URL as it might be using an ssh secret which does not use https://
-  TARGET_CLEAN_REPO=$(shell git ls-remote --get-url --symref $(TARGET_ORIGIN))
-  HELM_OPTS=-f values-global.yaml --set main.tokenSecret=$(TOKEN_SECRET) --set main.tokenSecretNamespace=$(TOKEN_NAMESPACE) --set main.git.repoURL="$(TARGET_CLEAN_REPO)" --set main.git.revision=$(TARGET_BRANCH) $(TARGET_SITE_OPT) $(UUID_HELM_OPTS) $(EXTRA_HELM_OPTS)
+  TARGET_REPO := $(TARGET_REPO_RAW)
+  SECRET_OPTS := --set main.tokenSecret=$(TOKEN_SECRET) --set main.tokenSecretNamespace=$(TOKEN_NAMESPACE)
 endif
 
+HELM_OPTS := -f values-global.yaml \
+             --set main.git.repoURL="$(TARGET_REPO)" \
+             --set main.git.revision=$(TARGET_BRANCH) \
+             $(SECRET_OPTS) \
+             $(TARGET_SITE_OPT) \
+             $(UUID_HELM_OPTS) \
+             $(EXTRA_HELM_OPTS)
+
 # Helm does the right thing and fetches all the tags and detects the newest one
 PATTERN_INSTALL_CHART ?= oci://quay.io/hybridcloudpatterns/pattern-install
 
@@ -138,8 +146,6 @@ token-kubeconfig: ## Create a local ~/.kube/config with password (not usually ne
 
 ##@ Validation Tasks
 
-# We only check the remote ssh git branch's existance if we're not running inside a container
-# as getting ssh auth working inside a container seems a bit brittle
 # If the main repoUpstreamURL field is set, then we need to check against
 # that and not target_repo
 .PHONY: validate-origin
@@ -250,16 +256,14 @@ super-linter: ## Runs super linter locally
 					-e VALIDATE_JSCPD=false \
 					-e VALIDATE_JSON_PRETTIER=false \
 					-e VALIDATE_MARKDOWN_PRETTIER=false \
-					-e VALIDATE_KUBERNETES_KUBECONFORM=false \
 					-e VALIDATE_PYTHON_PYLINT=false \
 					-e VALIDATE_SHELL_SHFMT=false \
-					-e VALIDATE_TEKTON=false \
 					-e VALIDATE_YAML=false \
 					-e VALIDATE_YAML_PRETTIER=false \
 					$(DISABLE_LINTERS) \
 					-v $(PWD):/tmp/lint:rw,z \
 					-w /tmp/lint \
-					ghcr.io/super-linter/super-linter:slim-v7
+					ghcr.io/super-linter/super-linter@sha256:6c71bd17ab38ceb7acb5b93ef72f5c2288b5456a5c82693ded3ee8bb501bba7f # slim-v8.1.0
 
 .PHONY: deploy upgrade legacy-deploy legacy-upgrade
 deploy upgrade legacy-deploy legacy-upgrade:

common/README.md

@@ -7,7 +7,7 @@
 This is the `main` branch of common and it assumes that the pattern is fully
 multisource (meaning that any used charts from VP is actually referenced from
 either a helm chart repository or quay repository). I.e. there are no helm
-charts contained in this branch of common and there is no ansible code neither.
+charts contained in this branch of common and there is no Ansible code neither.
 
 The helm charts now live in separate repositories under the VP
 [organization](https://github.com/validatedpatterns) on GitHub. The repositories are:
@@ -19,10 +19,10 @@ The helm charts now live in separate repositories under the VP
 - acm-chart
 - letsencrypt-chart
 
-The ansible bits live in this [repository](https://github.com/validatedpatterns/rhvp.cluster_utils)
+The Ansible bits live in this [repository](https://github.com/validatedpatterns/rhvp.cluster_utils)
 
 In order to be able to use this "slimmed-down" main branch of common you *must*
-use a 0.9.* clustergroup-chart that. Add the following to your `values-global.yaml`:
+use a 0.9.\* clustergroup-chart that. Add the following to your `values-global.yaml`:
 
 ```yaml
 main:
@@ -48,4 +48,4 @@ git merge -s subtree -Xtheirs -Xsubtree=common common-upstream/main
 
 ## Secrets
 
-There are two different secret formats parsed by the ansible bits. Both are documented [here](https://github.com/validatedpatterns/common/tree/main/ansible/roles/vault_utils/README.md)
+There are two different secret formats parsed by the Ansible bits. Both are documented [in the vault role](https://github.com/validatedpatterns/rhvp.cluster_utils/blob/main/roles/vault_utils/README.md)