🐛 [bug] - Argo CD does not sync the Jenkins DeploymentConfig after adding a new env variable was added

[rmarting]

📝 Description

The last part of the This is GitOps exercise is not successfully applied by Argo CD. Once the new environment variable is added, Argo CD identify the changed with the OutOfSync status, however, the reconcilization does not happen.

The final Application definition of Jenkins was:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"argoproj.io/v1alpha1","kind":"Application","metadata":{"annotations":{},"finalizers":["resources-finalizer.argocd.argoproj.io"],"labels":{"app.kubernetes.io/instance":"jenkins","rht-gitops.com/zteam-ci-cd":"ubiquitous-journey"},"name":"jenkins","namespace":"zteam-ci-cd"},"spec":{"destination":{"namespace":"zteam-ci-cd","server":"https://kubernetes.default.svc"},"ignoreDifferences":[{"group":"apps.openshift.io","jsonPointers":["/spec/replicas","/spec/template/spec/containers/0/image","/spec/triggers/0/imageChangeParams/lastTriggeredImage","/spec/triggers/1/imageChangeParams/lastTriggeredImage","/spec/template/metadata/annotations/rollme"],"kind":"DeploymentConfig"},{"group":"apps","jsonPointers":["/spec/replicas","/spec/template/spec/containers/0/image","/spec/template/spec/containers/1/image"],"kind":"Deployment"},{"group":"build.openshift.io","jsonPointers":["/spec/triggers"],"kind":"BuildConfig"},{"group":"route.openshift.io","jsonPointers":["/status/ingress"],"kind":"Route"},{"group":"integreatly.org","jsonPointers":["/spec/datasources/0/secureJsonData/httpHeaderValue1"],"kind":"GrafanaDataSource"}],"project":"default","source":{"helm":{"releaseName":"jenkins","values":"buildconfigs:\n- builder_image_kind: DockerImage\n  builder_image_name: quay.io/openshift/origin-jenkins\n  builder_image_tag: \"4.12\"\n  name: jenkins\n  source_context_dir: /\n  source_repo: https://github.com/rht-labs/s2i-config-jenkins.git\n  source_repo_ref: master\n  strategy_type: Source\n- name: jenkins-agent-npm\n- name: jenkins-agent-mvn\n- name: jenkins-agent-helm\n- name: jenkins-agent-argocd\ndeployment:\n  env_vars:\n  - name: GITLAB_DEFAULT_BRANCH\n    value: main\n  - name: GITLAB_HOST\n    value: \"\"\n  - name: GITLAB_GROUP_NAME\n    value: \"\"\n  - name: GIT_SSL_NO_VERIFY\n    value: \"true\"\n  - name: JAVA_ARGS\n    value: -Dorg.jenkinsci.plugins.gitclient.GitClient.untrustedSSL=true\n  - name: BISCUITS\n    value: jaffa-cakes\npersistence: false\nsource_secrets:\n- name: nexus-password\n  password: admin123\n  username: admin\n"},"path":"charts/jenkins","repoURL":"https://github.com/redhat-cop/helm-charts.git","targetRevision":"jenkins-1.0.10"},"syncPolicy":{"automated":{"prune":true,"selfHeal":true},"syncOptions":["Validate=true"]}}}
  resourceVersion: '7721817'
  name: jenkins
  uid: 3fa3d667-d8fb-47ac-9a27-7357dd27cecb
  creationTimestamp: '2024-09-12T07:57:13Z'
  generation: 13344
  managedFields:
    - apiVersion: argoproj.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:annotations':
            .: {}
            'f:kubectl.kubernetes.io/last-applied-configuration': {}
          'f:finalizers':
            .: {}
            'v:"resources-finalizer.argocd.argoproj.io"': {}
          'f:labels':
            .: {}
            'f:app.kubernetes.io/instance': {}
            'f:rht-gitops.com/zteam-ci-cd': {}
        'f:spec':
          .: {}
          'f:destination':
            .: {}
            'f:namespace': {}
            'f:server': {}
          'f:ignoreDifferences': {}
          'f:project': {}
          'f:source':
            .: {}
            'f:helm':
              .: {}
              'f:releaseName': {}
              'f:values': {}
            'f:path': {}
            'f:repoURL': {}
            'f:targetRevision': {}
          'f:syncPolicy':
            .: {}
            'f:automated':
              .: {}
              'f:prune': {}
              'f:selfHeal': {}
            'f:syncOptions': {}
      manager: argocd-controller
      operation: Update
      time: '2024-09-12T07:57:13Z'
    - apiVersion: argoproj.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:status':
          'f:history': {}
          'f:sourceType': {}
          'f:controllerNamespace': {}
          'f:conditions': {}
          .: {}
          'f:sync':
            .: {}
            'f:comparedTo':
              .: {}
              'f:destination':
                .: {}
                'f:namespace': {}
                'f:server': {}
              'f:ignoreDifferences': {}
              'f:source':
                .: {}
                'f:helm':
                  .: {}
                  'f:releaseName': {}
                  'f:values': {}
                'f:path': {}
                'f:repoURL': {}
                'f:targetRevision': {}
            'f:revision': {}
            'f:status': {}
          'f:health':
            .: {}
            'f:status': {}
          'f:resources': {}
          'f:summary':
            .: {}
            'f:images': {}
          'f:reconciledAt': {}
          'f:operationState':
            .: {}
            'f:finishedAt': {}
            'f:message': {}
            'f:operation':
              .: {}
              'f:initiatedBy':
                .: {}
                'f:automated': {}
              'f:retry':
                .: {}
                'f:limit': {}
              'f:sync':
                .: {}
                'f:prune': {}
                'f:resources': {}
                'f:revision': {}
                'f:syncOptions': {}
            'f:phase': {}
            'f:retryCount': {}
            'f:startedAt': {}
            'f:syncResult':
              .: {}
              'f:resources': {}
              'f:revision': {}
              'f:source':
                .: {}
                'f:helm':
                  .: {}
                  'f:releaseName': {}
                  'f:values': {}
                'f:path': {}
                'f:repoURL': {}
                'f:targetRevision': {}
      manager: argocd-application-controller
      operation: Update
      time: '2024-09-13T05:58:51Z'
  namespace: zteam-ci-cd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
  labels:
    app.kubernetes.io/instance: jenkins
    rht-gitops.com/zteam-ci-cd: ubiquitous-journey
spec:
  destination:
    namespace: zteam-ci-cd
    server: 'https://kubernetes.default.svc'
  ignoreDifferences:
    - group: apps.openshift.io
      jsonPointers:
        - /spec/replicas
        - /spec/template/spec/containers/0/image
        - /spec/triggers/0/imageChangeParams/lastTriggeredImage
        - /spec/triggers/1/imageChangeParams/lastTriggeredImage
        - /spec/template/metadata/annotations/rollme
      kind: DeploymentConfig
    - group: apps
      jsonPointers:
        - /spec/replicas
        - /spec/template/spec/containers/0/image
        - /spec/template/spec/containers/1/image
      kind: Deployment
    - group: build.openshift.io
      jsonPointers:
        - /spec/triggers
      kind: BuildConfig
    - group: route.openshift.io
      jsonPointers:
        - /status/ingress
      kind: Route
    - group: integreatly.org
      jsonPointers:
        - /spec/datasources/0/secureJsonData/httpHeaderValue1
      kind: GrafanaDataSource
  project: default
  source:
    helm:
      releaseName: jenkins
      values: |
        buildconfigs:
        - builder_image_kind: DockerImage
          builder_image_name: quay.io/openshift/origin-jenkins
          builder_image_tag: "4.12"
          name: jenkins
          source_context_dir: /
          source_repo: https://github.com/rht-labs/s2i-config-jenkins.git
          source_repo_ref: master
          strategy_type: Source
        - name: jenkins-agent-npm
        - name: jenkins-agent-mvn
        - name: jenkins-agent-helm
        - name: jenkins-agent-argocd
        deployment:
          env_vars:
          - name: GITLAB_DEFAULT_BRANCH
            value: main
          - name: GITLAB_HOST
            value: ""
          - name: GITLAB_GROUP_NAME
            value: ""
          - name: GIT_SSL_NO_VERIFY
            value: "true"
          - name: JAVA_ARGS
            value: -Dorg.jenkinsci.plugins.gitclient.GitClient.untrustedSSL=true
          - name: BISCUITS
            value: jaffa-cakes
        persistence: false
        source_secrets:
        - name: nexus-password
          password: admin123
          username: admin
    path: charts/jenkins
    repoURL: 'https://github.com/redhat-cop/helm-charts.git'
    targetRevision: jenkins-1.0.10
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - Validate=true
status:
  resources:
    - kind: Secret
      name: nexus-password
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - health:
        status: Healthy
      kind: Service
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - health:
        status: Healthy
      kind: Service
      name: jenkins-jnlp
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - kind: ServiceAccount
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: apps.openshift.io
      health:
        message: replication controller successfully rolled out
        status: Healthy
      kind: DeploymentConfig
      name: jenkins
      namespace: zteam-ci-cd
      status: OutOfSync
      version: v1
    - group: build.openshift.io
      kind: BuildConfig
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: build.openshift.io
      kind: BuildConfig
      name: jenkins-agent-argocd
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: build.openshift.io
      kind: BuildConfig
      name: jenkins-agent-helm
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: build.openshift.io
      kind: BuildConfig
      name: jenkins-agent-mvn
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: build.openshift.io
      kind: BuildConfig
      name: jenkins-agent-npm
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: image.openshift.io
      kind: ImageStream
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: image.openshift.io
      kind: ImageStream
      name: jenkins-agent-argocd
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: image.openshift.io
      kind: ImageStream
      name: jenkins-agent-helm
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: image.openshift.io
      kind: ImageStream
      name: jenkins-agent-mvn
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: image.openshift.io
      kind: ImageStream
      name: jenkins-agent-npm
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: rbac.authorization.k8s.io
      kind: RoleBinding
      name: jenkins-edit
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: route.openshift.io
      health:
        message: Route is healthy
        status: Healthy
      kind: Route
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
  summary:
    images:
      - 'image-registry.openshift-image-registry.svc:5000/zteam-ci-cd/jenkins@sha256:a58d5934960af5c473c3671ce4a8c80e2fdb291f2b21ab5ce6b3c055509a19bf'
      - 'quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:096b7ba83dcbd84e66a109a3118fc6af243ffad18fd46cc29d87bf3500954366'
      - 'quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:86cc0646b97bf8dfcd84a396a8301d6665d544bd7fb096cd88fcf24784703236'
  reconciledAt: '2024-09-13T05:58:51Z'
  operationState:
    finishedAt: '2024-09-12T13:11:56Z'
    message: 'one or more objects failed to apply, reason: error validating data: failed to check CRD: failed to list CRDs: the server was unable to return a response in the time allotted, but may still be processing the request (retried 5 times).'
    operation:
      initiatedBy:
        automated: true
      retry:
        limit: 5
      sync:
        prune: true
        resources:
          - group: apps.openshift.io
            kind: DeploymentConfig
            name: jenkins
        revision: 18856824beef2d7920e29746894aa3f593b4e60a
        syncOptions:
          - Validate=true
    phase: Failed
    retryCount: 5
    startedAt: '2024-09-12T12:55:51Z'
    syncResult:
      resources:
        - syncPhase: Sync
          message: 'error validating data: failed to check CRD: failed to list CRDs: the server was unable to return a response in the time allotted, but may still be processing the request'
          name: jenkins
          status: SyncFailed
          kind: DeploymentConfig
          version: v1
          hookPhase: Failed
          namespace: zteam-ci-cd
          group: apps.openshift.io
      revision: 18856824beef2d7920e29746894aa3f593b4e60a
      source:
        helm:
          releaseName: jenkins
          values: |
            buildconfigs:
            - builder_image_kind: DockerImage
              builder_image_name: quay.io/openshift/origin-jenkins
              builder_image_tag: "4.12"
              name: jenkins
              source_context_dir: /
              source_repo: https://github.com/rht-labs/s2i-config-jenkins.git
              source_repo_ref: master
              strategy_type: Source
            - name: jenkins-agent-npm
            - name: jenkins-agent-mvn
            - name: jenkins-agent-helm
            - name: jenkins-agent-argocd
            deployment:
              env_vars:
              - name: GITLAB_DEFAULT_BRANCH
                value: main
              - name: GITLAB_HOST
                value: ""
              - name: GITLAB_GROUP_NAME
                value: ""
              - name: GIT_SSL_NO_VERIFY
                value: "true"
              - name: JAVA_ARGS
                value: -Dorg.jenkinsci.plugins.gitclient.GitClient.untrustedSSL=true
              - name: BISCUITS
                value: jaffa-cakes
            persistence: false
            source_secrets:
            - name: nexus-password
              password: admin123
              username: admin
        path: charts/jenkins
        repoURL: 'https://github.com/redhat-cop/helm-charts.git'
        targetRevision: jenkins-1.0.10
  history:
    - deployStartedAt: '2024-09-12T07:57:14Z'
      deployedAt: '2024-09-12T07:57:20Z'
      id: 0
      initiatedBy:
        automated: true
      revision: 18856824beef2d7920e29746894aa3f593b4e60a
      source:
        helm:
          releaseName: jenkins
          values: |
            buildconfigs:
            - builder_image_kind: DockerImage
              builder_image_name: quay.io/openshift/origin-jenkins
              builder_image_tag: "4.12"
              name: jenkins
              source_context_dir: /
              source_repo: https://github.com/rht-labs/s2i-config-jenkins.git
              source_repo_ref: master
              strategy_type: Source
            - name: jenkins-agent-npm
            - name: jenkins-agent-mvn
            - name: jenkins-agent-helm
            - name: jenkins-agent-argocd
            deployment:
              env_vars:
              - name: GITLAB_DEFAULT_BRANCH
                value: main
              - name: GITLAB_HOST
                value: ""
              - name: GITLAB_GROUP_NAME
                value: ""
              - name: GIT_SSL_NO_VERIFY
                value: "true"
              - name: JAVA_ARGS
                value: -Dorg.jenkinsci.plugins.gitclient.GitClient.untrustedSSL=true
              - name: BISCUITS
                value: jaffa-cakes
            persistence: false
            source_secrets:
            - name: nexus-password
              password: admin123
              username: admin
        path: charts/jenkins
        repoURL: 'https://github.com/redhat-cop/helm-charts.git'
        targetRevision: jenkins-1.0.10
  sourceType: Helm
  conditions:
    - lastTransitionTime: '2024-09-12T13:11:56Z'
      message: 'Failed sync attempt to 18856824beef2d7920e29746894aa3f593b4e60a: one or more objects failed to apply, reason: error validating data: failed to check CRD: failed to list CRDs: the server was unable to return a response in the time allotted, but may still be processing the request (retried 5 times).'
      type: SyncError
  sync:
    comparedTo:
      destination:
        namespace: zteam-ci-cd
        server: 'https://kubernetes.default.svc'
      ignoreDifferences:
        - group: apps.openshift.io
          jsonPointers:
            - /spec/replicas
            - /spec/template/spec/containers/0/image
            - /spec/triggers/0/imageChangeParams/lastTriggeredImage
            - /spec/triggers/1/imageChangeParams/lastTriggeredImage
            - /spec/template/metadata/annotations/rollme
          kind: DeploymentConfig
        - group: apps
          jsonPointers:
            - /spec/replicas
            - /spec/template/spec/containers/0/image
            - /spec/template/spec/containers/1/image
          kind: Deployment
        - group: build.openshift.io
          jsonPointers:
            - /spec/triggers
          kind: BuildConfig
        - group: route.openshift.io
          jsonPointers:
            - /status/ingress
          kind: Route
        - group: integreatly.org
          jsonPointers:
            - /spec/datasources/0/secureJsonData/httpHeaderValue1
          kind: GrafanaDataSource
      source:
        helm:
          releaseName: jenkins
          values: |
            buildconfigs:
            - builder_image_kind: DockerImage
              builder_image_name: quay.io/openshift/origin-jenkins
              builder_image_tag: "4.12"
              name: jenkins
              source_context_dir: /
              source_repo: https://github.com/rht-labs/s2i-config-jenkins.git
              source_repo_ref: master
              strategy_type: Source
            - name: jenkins-agent-npm
            - name: jenkins-agent-mvn
            - name: jenkins-agent-helm
            - name: jenkins-agent-argocd
            deployment:
              env_vars:
              - name: GITLAB_DEFAULT_BRANCH
                value: main
              - name: GITLAB_HOST
                value: ""
              - name: GITLAB_GROUP_NAME
                value: ""
              - name: GIT_SSL_NO_VERIFY
                value: "true"
              - name: JAVA_ARGS
                value: -Dorg.jenkinsci.plugins.gitclient.GitClient.untrustedSSL=true
              - name: BISCUITS
                value: jaffa-cakes
            persistence: false
            source_secrets:
            - name: nexus-password
              password: admin123
              username: admin
        path: charts/jenkins
        repoURL: 'https://github.com/redhat-cop/helm-charts.git'
        targetRevision: jenkins-1.0.10
    revision: 18856824beef2d7920e29746894aa3f593b4e60a
    status: OutOfSync
  health:
    status: Healthy
  controllerNamespace: zteam-ci-cd

🚶 Steps to reproduce

Just follow the instructions on the latest version of OCP cluster created for this enablement.

OCP Version: 4.15.29
OpenShift Pipelines Operator Version: 1.13.1

🧙‍♀️ Suggested solution

There is a workaround that it is basically force the sync from Argo CD checking the Replace option for the DeploymentConfig. However, it should be Argo CD how applied automatically the change. After execute that action the Application definition of Jenkins was:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"argoproj.io/v1alpha1","kind":"Application","metadata":{"annotations":{},"finalizers":["resources-finalizer.argocd.argoproj.io"],"labels":{"app.kubernetes.io/instance":"jenkins","rht-gitops.com/zteam-ci-cd":"ubiquitous-journey"},"name":"jenkins","namespace":"zteam-ci-cd"},"spec":{"destination":{"namespace":"zteam-ci-cd","server":"https://kubernetes.default.svc"},"ignoreDifferences":[{"group":"apps.openshift.io","jsonPointers":["/spec/replicas","/spec/template/spec/containers/0/image","/spec/triggers/0/imageChangeParams/lastTriggeredImage","/spec/triggers/1/imageChangeParams/lastTriggeredImage","/spec/template/metadata/annotations/rollme"],"kind":"DeploymentConfig"},{"group":"apps","jsonPointers":["/spec/replicas","/spec/template/spec/containers/0/image","/spec/template/spec/containers/1/image"],"kind":"Deployment"},{"group":"build.openshift.io","jsonPointers":["/spec/triggers"],"kind":"BuildConfig"},{"group":"route.openshift.io","jsonPointers":["/status/ingress"],"kind":"Route"},{"group":"integreatly.org","jsonPointers":["/spec/datasources/0/secureJsonData/httpHeaderValue1"],"kind":"GrafanaDataSource"}],"project":"default","source":{"helm":{"releaseName":"jenkins","values":"buildconfigs:\n- builder_image_kind: DockerImage\n  builder_image_name: quay.io/openshift/origin-jenkins\n  builder_image_tag: \"4.12\"\n  name: jenkins\n  source_context_dir: /\n  source_repo: https://github.com/rht-labs/s2i-config-jenkins.git\n  source_repo_ref: master\n  strategy_type: Source\n- name: jenkins-agent-npm\n- name: jenkins-agent-mvn\n- name: jenkins-agent-helm\n- name: jenkins-agent-argocd\ndeployment:\n  env_vars:\n  - name: GITLAB_DEFAULT_BRANCH\n    value: main\n  - name: GITLAB_HOST\n    value: \"\"\n  - name: GITLAB_GROUP_NAME\n    value: \"\"\n  - name: GIT_SSL_NO_VERIFY\n    value: \"true\"\n  - name: JAVA_ARGS\n    value: -Dorg.jenkinsci.plugins.gitclient.GitClient.untrustedSSL=true\n  - name: BISCUITS\n    value: jaffa-cakes\npersistence: false\nsource_secrets:\n- name: nexus-password\n  password: admin123\n  username: admin\n"},"path":"charts/jenkins","repoURL":"https://github.com/redhat-cop/helm-charts.git","targetRevision":"jenkins-1.0.10"},"syncPolicy":{"automated":{"prune":true,"selfHeal":true},"syncOptions":["Validate=true"]}}}
  resourceVersion: '7725703'
  name: jenkins
  uid: 3fa3d667-d8fb-47ac-9a27-7357dd27cecb
  creationTimestamp: '2024-09-12T07:57:13Z'
  generation: 13352
  managedFields:
    - apiVersion: argoproj.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:annotations':
            .: {}
            'f:kubectl.kubernetes.io/last-applied-configuration': {}
          'f:finalizers':
            .: {}
            'v:"resources-finalizer.argocd.argoproj.io"': {}
          'f:labels':
            .: {}
            'f:app.kubernetes.io/instance': {}
            'f:rht-gitops.com/zteam-ci-cd': {}
        'f:spec':
          .: {}
          'f:destination':
            .: {}
            'f:namespace': {}
            'f:server': {}
          'f:ignoreDifferences': {}
          'f:project': {}
          'f:source':
            .: {}
            'f:helm':
              .: {}
              'f:releaseName': {}
              'f:values': {}
            'f:path': {}
            'f:repoURL': {}
            'f:targetRevision': {}
          'f:syncPolicy':
            .: {}
            'f:automated':
              .: {}
              'f:prune': {}
              'f:selfHeal': {}
            'f:syncOptions': {}
      manager: argocd-controller
      operation: Update
      time: '2024-09-12T07:57:13Z'
    - apiVersion: argoproj.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:status':
          'f:history': {}
          'f:sourceType': {}
          'f:controllerNamespace': {}
          .: {}
          'f:sync':
            .: {}
            'f:comparedTo':
              .: {}
              'f:destination':
                .: {}
                'f:namespace': {}
                'f:server': {}
              'f:ignoreDifferences': {}
              'f:source':
                .: {}
                'f:helm':
                  .: {}
                  'f:releaseName': {}
                  'f:values': {}
                'f:path': {}
                'f:repoURL': {}
                'f:targetRevision': {}
            'f:revision': {}
            'f:status': {}
          'f:health':
            .: {}
            'f:status': {}
          'f:resources': {}
          'f:summary':
            .: {}
            'f:images': {}
          'f:reconciledAt': {}
          'f:operationState':
            .: {}
            'f:finishedAt': {}
            'f:message': {}
            'f:operation':
              .: {}
              'f:initiatedBy':
                .: {}
                'f:username': {}
              'f:retry': {}
              'f:sync':
                .: {}
                'f:resources': {}
                'f:revision': {}
                'f:syncOptions': {}
                'f:syncStrategy':
                  .: {}
                  'f:hook': {}
            'f:phase': {}
            'f:startedAt': {}
            'f:syncResult':
              .: {}
              'f:resources': {}
              'f:revision': {}
              'f:source':
                .: {}
                'f:helm':
                  .: {}
                  'f:releaseName': {}
                  'f:values': {}
                'f:path': {}
                'f:repoURL': {}
                'f:targetRevision': {}
      manager: argocd-application-controller
      operation: Update
      time: '2024-09-13T06:03:02Z'
  namespace: zteam-ci-cd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
  labels:
    app.kubernetes.io/instance: jenkins
    rht-gitops.com/zteam-ci-cd: ubiquitous-journey
spec:
  destination:
    namespace: zteam-ci-cd
    server: 'https://kubernetes.default.svc'
  ignoreDifferences:
    - group: apps.openshift.io
      jsonPointers:
        - /spec/replicas
        - /spec/template/spec/containers/0/image
        - /spec/triggers/0/imageChangeParams/lastTriggeredImage
        - /spec/triggers/1/imageChangeParams/lastTriggeredImage
        - /spec/template/metadata/annotations/rollme
      kind: DeploymentConfig
    - group: apps
      jsonPointers:
        - /spec/replicas
        - /spec/template/spec/containers/0/image
        - /spec/template/spec/containers/1/image
      kind: Deployment
    - group: build.openshift.io
      jsonPointers:
        - /spec/triggers
      kind: BuildConfig
    - group: route.openshift.io
      jsonPointers:
        - /status/ingress
      kind: Route
    - group: integreatly.org
      jsonPointers:
        - /spec/datasources/0/secureJsonData/httpHeaderValue1
      kind: GrafanaDataSource
  project: default
  source:
    helm:
      releaseName: jenkins
      values: |
        buildconfigs:
        - builder_image_kind: DockerImage
          builder_image_name: quay.io/openshift/origin-jenkins
          builder_image_tag: "4.12"
          name: jenkins
          source_context_dir: /
          source_repo: https://github.com/rht-labs/s2i-config-jenkins.git
          source_repo_ref: master
          strategy_type: Source
        - name: jenkins-agent-npm
        - name: jenkins-agent-mvn
        - name: jenkins-agent-helm
        - name: jenkins-agent-argocd
        deployment:
          env_vars:
          - name: GITLAB_DEFAULT_BRANCH
            value: main
          - name: GITLAB_HOST
            value: ""
          - name: GITLAB_GROUP_NAME
            value: ""
          - name: GIT_SSL_NO_VERIFY
            value: "true"
          - name: JAVA_ARGS
            value: -Dorg.jenkinsci.plugins.gitclient.GitClient.untrustedSSL=true
          - name: BISCUITS
            value: jaffa-cakes
        persistence: false
        source_secrets:
        - name: nexus-password
          password: admin123
          username: admin
    path: charts/jenkins
    repoURL: 'https://github.com/redhat-cop/helm-charts.git'
    targetRevision: jenkins-1.0.10
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - Validate=true
status:
  resources:
    - kind: Secret
      name: nexus-password
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - health:
        status: Healthy
      kind: Service
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - health:
        status: Healthy
      kind: Service
      name: jenkins-jnlp
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - kind: ServiceAccount
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: apps.openshift.io
      health:
        message: replication controller successfully rolled out
        status: Healthy
      kind: DeploymentConfig
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: build.openshift.io
      kind: BuildConfig
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: build.openshift.io
      kind: BuildConfig
      name: jenkins-agent-argocd
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: build.openshift.io
      kind: BuildConfig
      name: jenkins-agent-helm
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: build.openshift.io
      kind: BuildConfig
      name: jenkins-agent-mvn
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: build.openshift.io
      kind: BuildConfig
      name: jenkins-agent-npm
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: image.openshift.io
      kind: ImageStream
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: image.openshift.io
      kind: ImageStream
      name: jenkins-agent-argocd
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: image.openshift.io
      kind: ImageStream
      name: jenkins-agent-helm
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: image.openshift.io
      kind: ImageStream
      name: jenkins-agent-mvn
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: image.openshift.io
      kind: ImageStream
      name: jenkins-agent-npm
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: rbac.authorization.k8s.io
      kind: RoleBinding
      name: jenkins-edit
      namespace: zteam-ci-cd
      status: Synced
      version: v1
    - group: route.openshift.io
      health:
        message: Route is healthy
        status: Healthy
      kind: Route
      name: jenkins
      namespace: zteam-ci-cd
      status: Synced
      version: v1
  summary:
    images:
      - 'image-registry.openshift-image-registry.svc:5000/zteam-ci-cd/jenkins@sha256:a58d5934960af5c473c3671ce4a8c80e2fdb291f2b21ab5ce6b3c055509a19bf'
      - 'quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:096b7ba83dcbd84e66a109a3118fc6af243ffad18fd46cc29d87bf3500954366'
      - 'quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:86cc0646b97bf8dfcd84a396a8301d6665d544bd7fb096cd88fcf24784703236'
  reconciledAt: '2024-09-13T06:01:46Z'
  operationState:
    finishedAt: '2024-09-13T06:01:46Z'
    message: successfully synced (all tasks run)
    operation:
      initiatedBy:
        username: user20
      retry: {}
      sync:
        resources:
          - group: apps.openshift.io
            kind: DeploymentConfig
            name: jenkins
            namespace: zteam-ci-cd
        revision: 18856824beef2d7920e29746894aa3f593b4e60a
        syncOptions:
          - Validate=true
          - Replace=true
        syncStrategy:
          hook: {}
    phase: Succeeded
    startedAt: '2024-09-13T06:01:46Z'
    syncResult:
      resources:
        - syncPhase: Sync
          message: deploymentconfig.apps.openshift.io/jenkins replaced
          name: jenkins
          status: Synced
          kind: DeploymentConfig
          version: v1
          hookPhase: Running
          namespace: zteam-ci-cd
          group: apps.openshift.io
      revision: 18856824beef2d7920e29746894aa3f593b4e60a
      source:
        helm:
          releaseName: jenkins
          values: |
            buildconfigs:
            - builder_image_kind: DockerImage
              builder_image_name: quay.io/openshift/origin-jenkins
              builder_image_tag: "4.12"
              name: jenkins
              source_context_dir: /
              source_repo: https://github.com/rht-labs/s2i-config-jenkins.git
              source_repo_ref: master
              strategy_type: Source
            - name: jenkins-agent-npm
            - name: jenkins-agent-mvn
            - name: jenkins-agent-helm
            - name: jenkins-agent-argocd
            deployment:
              env_vars:
              - name: GITLAB_DEFAULT_BRANCH
                value: main
              - name: GITLAB_HOST
                value: ""
              - name: GITLAB_GROUP_NAME
                value: ""
              - name: GIT_SSL_NO_VERIFY
                value: "true"
              - name: JAVA_ARGS
                value: -Dorg.jenkinsci.plugins.gitclient.GitClient.untrustedSSL=true
              - name: BISCUITS
                value: jaffa-cakes
            persistence: false
            source_secrets:
            - name: nexus-password
              password: admin123
              username: admin
        path: charts/jenkins
        repoURL: 'https://github.com/redhat-cop/helm-charts.git'
        targetRevision: jenkins-1.0.10
  history:
    - deployStartedAt: '2024-09-12T07:57:14Z'
      deployedAt: '2024-09-12T07:57:20Z'
      id: 0
      initiatedBy:
        automated: true
      revision: 18856824beef2d7920e29746894aa3f593b4e60a
      source:
        helm:
          releaseName: jenkins
          values: |
            buildconfigs:
            - builder_image_kind: DockerImage
              builder_image_name: quay.io/openshift/origin-jenkins
              builder_image_tag: "4.12"
              name: jenkins
              source_context_dir: /
              source_repo: https://github.com/rht-labs/s2i-config-jenkins.git
              source_repo_ref: master
              strategy_type: Source
            - name: jenkins-agent-npm
            - name: jenkins-agent-mvn
            - name: jenkins-agent-helm
            - name: jenkins-agent-argocd
            deployment:
              env_vars:
              - name: GITLAB_DEFAULT_BRANCH
                value: main
              - name: GITLAB_HOST
                value: ""
              - name: GITLAB_GROUP_NAME
                value: ""
              - name: GIT_SSL_NO_VERIFY
                value: "true"
              - name: JAVA_ARGS
                value: -Dorg.jenkinsci.plugins.gitclient.GitClient.untrustedSSL=true
              - name: BISCUITS
                value: jaffa-cakes
            persistence: false
            source_secrets:
            - name: nexus-password
              password: admin123
              username: admin
        path: charts/jenkins
        repoURL: 'https://github.com/redhat-cop/helm-charts.git'
        targetRevision: jenkins-1.0.10
  sourceType: Helm
  sync:
    comparedTo:
      destination:
        namespace: zteam-ci-cd
        server: 'https://kubernetes.default.svc'
      ignoreDifferences:
        - group: apps.openshift.io
          jsonPointers:
            - /spec/replicas
            - /spec/template/spec/containers/0/image
            - /spec/triggers/0/imageChangeParams/lastTriggeredImage
            - /spec/triggers/1/imageChangeParams/lastTriggeredImage
            - /spec/template/metadata/annotations/rollme
          kind: DeploymentConfig
        - group: apps
          jsonPointers:
            - /spec/replicas
            - /spec/template/spec/containers/0/image
            - /spec/template/spec/containers/1/image
          kind: Deployment
        - group: build.openshift.io
          jsonPointers:
            - /spec/triggers
          kind: BuildConfig
        - group: route.openshift.io
          jsonPointers:
            - /status/ingress
          kind: Route
        - group: integreatly.org
          jsonPointers:
            - /spec/datasources/0/secureJsonData/httpHeaderValue1
          kind: GrafanaDataSource
      source:
        helm:
          releaseName: jenkins
          values: |
            buildconfigs:
            - builder_image_kind: DockerImage
              builder_image_name: quay.io/openshift/origin-jenkins
              builder_image_tag: "4.12"
              name: jenkins
              source_context_dir: /
              source_repo: https://github.com/rht-labs/s2i-config-jenkins.git
              source_repo_ref: master
              strategy_type: Source
            - name: jenkins-agent-npm
            - name: jenkins-agent-mvn
            - name: jenkins-agent-helm
            - name: jenkins-agent-argocd
            deployment:
              env_vars:
              - name: GITLAB_DEFAULT_BRANCH
                value: main
              - name: GITLAB_HOST
                value: ""
              - name: GITLAB_GROUP_NAME
                value: ""
              - name: GIT_SSL_NO_VERIFY
                value: "true"
              - name: JAVA_ARGS
                value: -Dorg.jenkinsci.plugins.gitclient.GitClient.untrustedSSL=true
              - name: BISCUITS
                value: jaffa-cakes
            persistence: false
            source_secrets:
            - name: nexus-password
              password: admin123
              username: admin
        path: charts/jenkins
        repoURL: 'https://github.com/redhat-cop/helm-charts.git'
        targetRevision: jenkins-1.0.10
    revision: 18856824beef2d7920e29746894aa3f593b4e60a
    status: Synced
  health:
    status: Healthy
  controllerNamespace: zteam-ci-cd

[alvarolop]

Hello, there is an open bug in ArgoCD project about this topic. According to the issue, ArgoCD doesn't check differences inside arrays like env vars and annotations: argoproj/argo-cd#4537

Maybe it would be just easier to change the documentation to ask to scale to two replicas and see how this scales down automatically (It works out of the box)