Skip to content

GKE ValidatingAdmissionPolicy 'validating-node-p4sa-audience' CEL error blocks pod creation when Barman Cloud plugin is active #364

@ameijboom

Description

@ameijboom

We've recently deployed a new GKE cluster (v1.32.4) with CNPG v1.26. Standalone CNPG clusters (without cloud backup plugins) work correctly. However, when we attempt to deploy a CNPG Cluster with the barman-cloud.cloudnative-pg.io plugin enabled, pod creation fails. The CNPG operator logs show that the pod creation is denied by a GKE-managed ValidatingAdmissionPolicy named validating-node-p4sa-audience.

The denial is not because our pod is trying to use the forbidden GKE node agent token, but because the CEL expression within the ValidatingAdmissionPolicy itself errors during evaluation when processing the pod spec generated by CNPG.

2025-05-28 12:45:29.924Z {"level":"info","ts":"2025-05-28T12:45:29.923792366Z","msg":"Creating new Pod to reattach a PVC","controller":"cluster","controllerGroup":"postgresql.cnpg.io","controllerKind":"Cluster","Cluster":{"name":"shlink-db","namespace":"shlink"},"namespace":"shlink","name":"shlink-db","reconcileID":"1bb860b5-3a01-4b66-9635-5b754e84ecb6","pod":"shlink-db-2","pvc":"shlink-db-2"}
2025-05-28 12:45:29.976Z {"level":"error","ts":"2025-05-28T12:45:29.976336667Z","msg":"Reconciler error","controller":"cluster","controllerGroup":"postgresql.cnpg.io","controllerKind":"Cluster","Cluster":{"name":"shlink-db","namespace":"shlink"},"namespace":"shlink","name":"shlink-db","reconcileID":"1bb860b5-3a01-4b66-9635-5b754e84ecb6","error":"unable to create Pod: pods \"shlink-db-2\" is forbidden: ValidatingAdmissionPolicy 'validating-node-p4sa-audience' with binding 'validating-node-p4sa-audience-binding' denied request: expression '!variables.podSpec.?volumes.orValue([]).exists(\n  volume,\n  has(volume.projected) &&\n  volume.projected.?sources.orValue([]).exists(\n    source,\n    has(source.serviceAccountToken) &&\n    has(source.serviceAccountToken.audience) &&\n    source.serviceAccountToken.audience == \"https://container.googleapis.com/v1/projects/project-number/locations/location/clusters/cluster-name/generateClusterNodeAgentToken\"))' resulted in error: got 'types.Null', expected iterable type","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\tpkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:341\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\tpkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:288\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\tpkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:249"}

Specifically deploying with this part of the configuration causes the above error in the cnpg-operator pod:

  plugins:
  - name: barman-cloud.cloudnative-pg.io
    isWALArchiver: true
    parameters:
      barmanObjectName: google-store

The policy in question:

Name:         validating-node-p4sa-audience
Namespace:
Labels:       addonmanager.kubernetes.io/mode=Reconcile
Annotations:  components.gke.io/component-name: securitycontext
              components.gke.io/component-version: 1.2.0
              components.gke.io/layer: addon
API Version:  admissionregistration.k8s.io/v1
Kind:         ValidatingAdmissionPolicy
Metadata:
  Creation Timestamp:  2025-05-20T21:36:30Z
  Generation:          1
  Resource Version:    1747834241323599010
  UID:                 dd41e2fc-94c3-4fa5-a6d2-1a78f924cc2a
Spec:
  Failure Policy:  Fail
  Match Conditions:
    Expression:  ![ "system:addon-manager", "system:serviceaccount:kube-system:cronjob-controller", "system:serviceaccount:kube-system:daemon-set-controller", "system:serviceaccount:kube-system:deployment-controller", "system:serviceaccount:kube-system:job-controller", "system:serviceaccount:kube-system:replicaset-controller", "system:serviceaccount:kube-system:replication-controller", "system:serviceaccount:kube-system:statefulset-controller" ].exists(sa, sa == request.userInfo.username)
    Name:        is-not-allowlisted-identity
  Match Constraints:
    Match Policy:  Equivalent
    Namespace Selector:
    Object Selector:
    Resource Rules:
      API Groups:

      API Versions:
        v1
      Operations:
        CREATE
        UPDATE
      Resources:
        pods
        podtemplates
        replicationcontrollers
      Scope:  *
      API Groups:
        apps
      API Versions:
        v1
      Operations:
        CREATE
        UPDATE
      Resources:
        deployments
        replicasets
        statefulsets
        daemonsets
      Scope:  *
      API Groups:
        batch
      API Versions:
        v1
      Operations:
        CREATE
        UPDATE
      Resources:
        jobs
        cronjobs
      Scope:  *
  Validations:
    Expression:  !variables.podSpec.?volumes.orValue([]).exists(
  volume,
  has(volume.projected) &&
  volume.projected.?sources.orValue([]).exists(
    source,
    has(source.serviceAccountToken) &&
    has(source.serviceAccountToken.audience) &&
    source.serviceAccountToken.audience == "https://container.googleapis.com/v1/projects/<project-number>/locations/<location>/clusters/<cluster>/generateClusterNodeAgentToken"))
  Variables:
    Expression:  object.kind == "Pod" ? object.spec : (
  object.kind == "PodTemplate" ? object.template.spec : (
    object.kind == "CronJob" ?
      object.spec.jobTemplate.spec.template.spec : (
        ["Deployment","ReplicaSet","DaemonSet","StatefulSet","Job", "ReplicationController"].exists(kind, object.kind == kind) ? object.spec.template.spec : {}
      )
  )
)
    Name:  podSpec
Status:
  Observed Generation:  1
  Type Checking:
Events:  <none>

Any help would be much appreciated, as we'd like to adopt the new form of backups within our clusters.

GKE version: 1.32.4-gke.200 (Release channel: REGULAR)
CNPG version: v1.26.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions