OPA policy for fields which are defaulted if omitted

[mtheeren-asml]

Hi,

I'm trying to create a multitenant system and am writing a rego policy which says certain annotations / fields are not allowed to be present when creating a pod. I am also using Flux CD to deploy the manifests on my cluster.

Now, let's say for example we only allow the annotations app.kubernetes.io/name and app.kubernetes.io/instance to be present when specifying a manifest for a pod. Kubernetes will automatically add annotations from operators / webhooks / ... and will add defaults for fields which are omitted.

This means my final deployed pod will have for example these annotations:

gatekeeper.sh/mutation-id: b89dc9a7-cb7e-476d-bc99-a2a01ff4d00d
gatekeeper.sh/mutations: Assign//podspec-os:1, Assign//seccomp-pod-ephemeral:1, 
Assign//seccomp-pod-init:1, ModifySet//dropcapabilities-init:1, ModifySet//dropcapabilities:1 
k8s.v1.cni.cncf.io/networks: istio-cni     
kubectl.kubernetes.io/default-container: 643f8e4aafcd14cdd565668d                    
kubectl.kubernetes.io/default-logs-container: 643f8e4aafcd14cdd565668d

Is there any way to get only the initial fields which are in the manifest picked up by Flux, before any defaulting is added by Kubernetes and webhooks interfere with the object?

Thanks in advance!

[anderseknert]

The validating webhook step always comes after the mutating one, so if you'd want to read the resources before the mutating admission controllers have been applied, you'd likely need to do so in a mutating admission controller webhook yourself.. I'm not sure if it works or if it's a good idea though :)

Another option would be to do the check in the CI/CD pipeline... i.e. check the files on disk before they are deployed.

[mtheeren-asml]

Thanks for the quick reply!
So if I understand correctly there's no way to do it in the gatekeeper policy itself, right?
Doing it in CI/CD is indeed an option, will look into that.

[anderseknert]

I'm not sure to be honest. It could be that you can configure Gatekeeper as a mutating admission controller, but still have it "validate" configuration... but it sounds rather hacky :P

[mtheeren-asml]

True :p