Skip to content

Conversation

@mbaldessari
Copy link
Contributor

This PR moves to the latest common from the main branch. See https://validatedpatterns.io/blog/2024-09-26-slimming-of-common/
for more details

  • Bind mount /etc/pki in the wrapper
  • Properly error out in preview-all when we cannot connect to the cluster
  • Only include values files if they do exist in preview.sh
  • Do not error out in preview when kustomize: true
  • Also pass EXTRA_PLAYBOOK_OPTS environment setting
  • Use oc kustomize
  • Print out application being parsed for easier debugging
  • Add .global.secretStore.backend in the clustergroup schema
  • Fix check for secretStore.backend
  • Introduce a generic parameter override via the EXTRA_ARGS env variable
  • Allow overriding the oc lookups in preview.sh
  • Always include values-global.yaml and values-.yaml
  • Add help and message clarifying that preview has certain limits
  • Add clustergroup support to preview target
  • Fix preview when the application's index name is not the same as the name attribute
  • Force strings in extraParametersNested
  • In ACM policies do not use $ARGOCD_APP_SOURCE_ variables*
  • Add support for custom CAs
  • Update tests
  • Support for issue #459
  • Drop unused piece of schema json
  • Small cleanup to remove unneeded log messages
  • Support for cluster-wide proxy
  • Update ESO to 0.9.14
  • Update CRD from operator v0.0.44
  • Expose main.experimentalCapabilities in operator-install
  • Release clustergroup v0.8.3
  • feat: add support for hive clusterdeployments creating spokes
  • test: regenerated tests after clusterdeployment commit
  • Support remote repoURL when previewing templates
  • test: updated test-cased and regeneated expectations
  • Moved CLUSTERGROUP declaration to restore make preview-% functionality
  • Namespace argocd.argoproj.io/managed-by label issue
  • Fixed indenting and duplicate entries in application-policies.yaml
  • Add main.experimentalCapabilities to values.schema.json
  • Release clustergroup v0.8.4
  • Move the CLUSTERGROUP env variable for previews inside its section
  • Fix up tests after upstream PR merge
  • Drop old comments
  • bug: Fix to generate OperatorGroup definition when namespaces definition has labels
  • Try fallbacks for /etc/pki when it does not exist
  • Fix for multiple OperatorGroup rendering
  • Updated tests for CI
  • Update ESO to 0.9.16
  • Updated vault-helm to v0.28.0
  • Use --recurse-submodules when cloning
  • Drop support for IIB on OCP 4.12
  • Stop saving stderr in a file when running skopeo
  • Rearchitect the code so we can print out the actual error
  • Check for prereqs for IIB
  • chore: added annotations controling gitops and fail for missing meta for clusterdeployments
  • chore: removed managedclusterset spec
  • Rework IIB loading support
  • Switch to gitops-1.12
  • Add overrides for MCE IIB
  • Update README for IIB changes
  • Release clustergroup v0.8.5
  • New global scope argocdServer section for values-global.yaml
  • Improve readme for ACM IIB
  • Drop gitopsspec from pattern's CR
  • Allow customizing the VP operator subscription
  • Add retries when checking oc version
  • Add an imperative-admin-sa service account
  • Added support to label/annotate nodes
  • Added support to enable user workloads in control plane nodes
  • Added full support for the scheduler
  • Simplified PR for auto approve install plans
  • Switch to registry.redhat.io for the initContainer image
  • Update for ACM chart to application-policies.yaml
  • Add extraParameters to values.schema.json
  • Update ESO to 0.9.18
  • Update vault to 1.16.2
  • Feat: Followup to definition of extraParameters under the main section of a values file.
  • Use golang-external-secrets for the acm hub-ca bits
  • Only do the acm hub ca policy when vault is the backend
  • fix: when using clusterdeployments, secrets should exist in the cluster-namespace
  • Force rolebindings as early as possible
  • bug: Invalid OperatorGroup generated when ommitting targetNamespaces
  • Fix CI issue
  • Actually use adminServiceAccountName for the auto approve job
  • Make sure that the if condition on chart split is not always true
  • Bump super-linter from 5 to 6
  • Drop some validations for now
  • Add some debugging to the chart split action
  • Use a specific git version when running git subtree split
  • Release clustergroup v0.8.6
  • Add a sudo to apt-get command
  • Add some READMEs in the individual charts
  • Fix super-linter issues and upgrade local super-linter target
  • Skip unreachable spokes when setting up vault
  • Add no_log to spokes initialization task
  • Drop initContainers variable and make it the default
  • Update tests after dropping initContainers
  • Release clustergroup v0.8.7
  • Upgrade ESO to v0.9.19
  • Update helm version in CI to 3.14.0
  • Drop imperative.initcontainers.gitinit
  • Have unseal work whenever we specifiy HEAD
  • make resourceExclusion configurable
  • Update tests after upgrading resourceExclusions tweak
  • Make resourcehealthchecks configurable
  • Update tests after upgrading resourceHealthChecks change
  • Do not quote $BRANCH variable
  • Fix initcontainer race on spokes
  • Release clustergroup v0.8.8
  • configure annotation based resource tracking
  • Drop imperative.volumes and imperative.volumemounts
  • Properly quote vault kv command
  • Make HUB work when spokes point to in hub cluster gitea
  • update tests
  • ACM chart version 0.0.2
  • golang-external-secrets chart version 0.0.4
  • hashicorp-vault chart version 0.0.2
  • Implement multi-source
  • Add tests for proper multisource support on spokes
  • Release clustergroup v0.8.9
  • Fix multisource indent error
  • Release clustergroup v0.8.10
  • Release acm v0.0.3
  • Fix missing chart field
  • Release acm v0.0.4
  • Update chart versions
  • Update vault-helm to v0.28.1 and vault to 1.17.2
  • Update ESO to 0.9.20
  • Release new hashicorp-vault and golang-external-secrets charts
  • Add Ansible playbook
  • Put the playbook in a more normal location for us
  • Exclude new playbook
  • Allow for choice in where file is written
  • Fix pki bind mount when using podman machine
  • Allow originRepo to be set via make install and main.git parameters
  • resolves fix: add missing field to passwd entry #21 rag-llm-gitops
  • Fix vars that were erroneously dropped
  • Allow more flexibility with multiSourceConfig schema
  • Inject VALUES_SECRET env var
  • Add helmRepoUrl variable
  • Update letsencrypt to v0.1.1
  • Allow overriding gitops source on spokes
  • Update acm chart to v0.1.1
  • Update clustergroup chart to 0.8.11
  • Extend the schema for disconnected
  • Update clustergroup chart to 0.8.12
  • Update super-linter to v7
  • Fix action path
  • Fix action path v2
  • Upgrade ESO to v0.10.0
  • Upgrade vault to 1.17.3
  • Update hashicorp-vault to 0.1.2
  • Update golang-external-secrets to 0.1.2
  • Switch to gitops-1.13
  • Add a pushsecrets policy and vault path for ESO syncing
  • Fix PyInk warnings
  • Replace tabs with spaces and add some missing env vars that could be passed
  • Also push any changes to operator-install to its own repo
  • Update CRD from operator v0.0.55
  • Fix chart name in pattern-install branch+split wf
  • Tweak readme
  • Use $group.name in clusterset metadata
  • Expose originURL as helm value
  • Update tests after common rebase
  • Release clustergroup v0.8.13
  • Release acm v0.1.2
  • Drop schema required under the Main section
  • Release clustergroup v0.8.14
  • Update README for the v1 common branch
  • Drop all the helm charts
  • README fixes
  • Drop the operator-install folder (pattern-install chart)
  • Start using the OCI chart in lieu of operator-install
  • Drop the chart splitting workflows
  • Drop the test folder
  • Drop the json schema checking job
  • Add a README note on the main branch
  • Add a check to see if multisource for clustergroup is enabled
  • Move the common slim + multisource test into Makefile
  • Drop last bit of operator-install and the common symlink
  • Drop reference-output.yaml as it makes little sense in the slimmed common
  • Remove ansible code in favor of external collection
  • Make adjustments for ansible code moving into separate collection repo
  • Allow customizing the clusterGroupChartVersion
  • Release pattern-install v0.0.2
  • Drop the chart splitting workflows
  • Make sure we compare lists with lists when doing a difference
  • Add a note about the changes in common
  • Add qe-tests target
  • Remove assorted detritus
  • We don't have charts to test anymore
  • Drop lint.sh and its target in Makefile
  • Drop test.sh and its target in Makefile and also kubeconform
  • Read PATTERN_INSTALL_CHART from the env as well if it exists
  • Update secret backend change script
  • Update preview script
  • Introduce PATTERN_DISCONNECTED_HOME
  • Update Makefile to reflect a warning if no storageclass is present
  • Fix iib loading playbook name
  • Improve validate-origin
  • Make the make install output less daunting
  • Record the exit code at the right time
  • Fix path when invoking the qe run_test.sh script
  • Add a warning when pattern folder and pattern name differ
  • Drop some old and unused prereq test
  • Fix validated-schema make target
  • Wait some more before giving up
  • Fix argo-healthcheck target
  • Push TARGET_SITE into the in-container variables
  • Do not bind mount /etc/pki blindly
  • Switch to multisource charts
  • Drop old test diffs

mbaldessari and others added 30 commits May 17, 2024 10:28
…n of a values file.

- The operator adds these extraParameters to the extraParametersNested section as key/value pairs in the Cluster Wide ArgoCD Application created by the Validated Patterns operator.
- This update will add the user defined extra parameters on the ArgoCD Applications on the Spoke Clusters.

efinition of extraParameters under the main
We'd like to make the imperative namespace optional,
so let's use the golang-external-secrets one, which is probably
more correct anyways since the acm hub ca is tied to ESO anyways.
The acm hub ca is needed for ESO on spokes to connect to the vault on
the hub, there is no need for this when vault is not used, so let's
drop it in that case
Feat: Followup to definition of extraParameters under the main section of a values file.
…er-namespace

Co-authored-by: Michele Baldessari <[email protected]>
Co-authored-by: Alejandro Villegas <[email protected]>
Signed-off-by: Tomer Figenblat <[email protected]>
This is important because in some situations (we've observed this on the
clusterwide argo instance on spokes) the permissions are not there yet
when argo tries to create service accounts for the imperative SAs.

This means that the very first sync works up to the service account
creation which then fails due to lacking RBACs. This triggers a gitops
issue where selfheal never retries because the previous run failed and
so the app is in a stuck loop forever

Co-Authored-By: Jonny Rickard <[email protected]>

Closes: GITOPS-4677
Force rolebindings as early as possible
Problem Statement:
When setting a namespace like this:
    - openshift-distributed-tracing:
        operatorGroup: true
        targetNamespaces: []

The chart generates the following yaml:
```yaml
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-distributed-tracing-operator-group
  namespace: openshift-distributed-tracing
spec:
  targetNamespaces:
```

Which k8s rejects the targetNamespaces key as invalid when it attempts to apply it and removes it since it doesn't have a value, which just so happens to have the desired result of not setting the targetNamespaces (or a selector) to enable it for All Namespaces.
bug: Invalid OperatorGroup generated when omitting targetNamespaces
feat: use hive clusterdeployment for creating spoke clusters
…sters

Added support to control the scheduler/cluster spec
Actually use adminServiceAccountName for the auto approve job
This should fix the fact that jobs are triggered on unrelated changes
Make sure that the if condition on chart split is not always true
Add some debugging to the chart split action
Otherwise we will get errors during the command as git subtree has
regressed since v2.44.0
mbaldessari and others added 19 commits November 25, 2024 12:08
Fix path when invoking the qe run_test.sh script
Since the slimming of common this won't work anymore:

❯ make validate-schema
make -f common/Makefile validate-schema
make[1]: Entering directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops'
Validating clustergroup schema of:  ./values-global.yamlError: repo common not found
make[1]: *** [common/Makefile:162: validate-schema] Error 1
make[1]: Leaving directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops'
make: *** [Makefile:12: validate-schema] Error 2

Fix this to use the oci helm chart:
❯ make validate-schema
make -f common/Makefile validate-schema
make[1]: Entering directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops'
Validating clustergroup schema of:  ./values-global.yamlPulled: quay.io/hybridcloudpatterns/clustergroup:0.9.13
Digest: sha256:725af54c0a5ad8c2235676bbff2785ece62c9929ab58aaf33837aa3f19708ce6
 ./values-group-one.yamlPulled: quay.io/hybridcloudpatterns/clustergroup:0.9.13
Digest: sha256:725af54c0a5ad8c2235676bbff2785ece62c9929ab58aaf33837aa3f19708ce6
 ./values-hub.yamlPulled: quay.io/hybridcloudpatterns/clustergroup:0.9.13
Digest: sha256:725af54c0a5ad8c2235676bbff2785ece62c9929ab58aaf33837aa3f19708ce6

make[1]: Leaving directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops'
Yukin observed a case on a baremetal server where the install failed
with:

    make -f common/Makefile operator-deploy
    make[1]: Entering directory '/home/fedora/validated_patterns/multicloud-gitops'
    Checking repository:
      https://github.com/validatedpatterns-workspace/multicloud-gitops - branch 'qe_test-18760': OK
    Checking cluster:
      cluster-info: OK
      storageclass: OK
    Installing pattern: ....Installation failed [5/5]. Error:
    WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/fedora/rhvpsno2-intel-18760/auth/kubeconfig
    WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/fedora/rhvpsno2-intel-18760/auth/kubeconfig
    Pulled: quay.io/hybridcloudpatterns/pattern-install:0.0.7
    Digest: sha256:b845f86c735478cfd44b0b43842697851cec64737c737bd18a872fa86bb0484d
    customresourcedefinition.apiextensions.k8s.io/patterns.gitops.hybrid-cloud-patterns.io unchanged
    configmap/patterns-operator-config unchanged
    pattern.gitops.hybrid-cloud-patterns.io/multicloud-gitops created
    subscription.operators.coreos.com/patterns-operator unchanged
    make[1]: *** [common/Makefile:71: operator-deploy] Error 1
    make[1]: Leaving directory '/home/fedora/validated_patterns/multicloud-gitops'
    make: *** [Makefile:12: operator-deploy] Error 2

In fact the install proceeded just okay, we just gave up too early.
Let's double the amount of times we wait for this and also increase the
wait in between tries by 5 seconds. Hopefully this should cover these
edge cases.
If ACM is installed the search for `applications` matches the ACM one
and not the argo one.
This way we can override the TARGET_SITE when invoking pattern.sh
Add TARGET_SITE as an env variable
Since ubuntu sometimes has /etc/pki/fwupd with little else in there,
let's just bind mount /etc/pki when /etc/pki/tls exists.
This keeps Fedora-based distros running and should fix this specific
corner case observed on ubuntu.

Co-Authored-By: Akos Eros <[email protected]>

Closes: validatedpatterns/medical-diagnosis#130
Do not bind mount /etc/pki blindly
Moved to slimmed down common. Tested there is no diff from upstream:

$ diff -urN -x '.git' common ../common
$
@mbaldessari mbaldessari requested a review from day0hero February 8, 2025 11:59
@sauagarwa sauagarwa merged commit b029ad5 into validatedpatterns:main Mar 25, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants