Skip to content

Commit 3a358ed

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #45552 from jiahuif-forks/blog/validating-admission-policy/ga
Blog: ValidatingAdmissionPolicy GA
2 parents 2580f86 + 7b41c4b commit 3a358ed

File tree

2 files changed

+377
-0
lines changed

2 files changed

+377
-0
lines changed

‎content/en/blog/_posts/2024-04-24-validating-admission-policy-ga/index.md

Lines changed: 287 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,287 @@
1+
------
2+
layout: blog
3+
title: "Kubernetes 1.30: Validating Admission Policy Is Generally Available"
4+
slug: validating-admission-policy-ga
5+
date: 2024-04-24
6+
---
7+
8+
**Author:** Jiahui Feng (Google)
9+
10+
On behalf of the Kubernetes project, I am excited to announce that ValidatingAdmissionPolicy has reached
11+
**general availability**
12+
as part of Kubernetes 1.30 release. If you have not yet read about this new declarative alternative to
13+
validating admission webhooks, it may be interesting to read our
14+
[previous post](/blog/2022/12/20/validating-admission-policies-alpha/) about the new feature.
15+
If you have already heard about ValidatingAdmissionPolicies and you are eager to try them out,
16+
there is no better time to do it than now.
17+
18+
Let's have a taste of a ValidatingAdmissionPolicy, by replacing a simple webhook.
19+
20+
## Example admission webhook
21+
First, let's take a look at an example of a simple webhook. Here is an excerpt from a webhook that
22+
enforces `runAsNonRoot`, `readOnlyRootFilesystem`, `allowPrivilegeEscalation`, and `privileged` to be set to the least permissive values.
23+
24+
```go
25+
func verifyDeployment(deploy *appsv1.Deployment) error {
26+
var errs []error
27+
for i, c := range deploy.Spec.Template.Spec.Containers {
28+
if c.Name == "" {
29+
return fmt.Errorf("container %d has no name", i)
30+
}
31+
if c.SecurityContext == nil {
32+
errs = append(errs, fmt.Errorf("container %q does not have SecurityContext", c.Name))
33+
}
34+
if c.SecurityContext.RunAsNonRoot == nil || !*c.SecurityContext.RunAsNonRoot {
35+
errs = append(errs, fmt.Errorf("container %q must set RunAsNonRoot to true in its SecurityContext", c.Name))
36+
}
37+
if c.SecurityContext.ReadOnlyRootFilesystem == nil || !*c.SecurityContext.ReadOnlyRootFilesystem {
38+
errs = append(errs, fmt.Errorf("container %q must set ReadOnlyRootFilesystem to true in its SecurityContext", c.Name))
39+
}
40+
if c.SecurityContext.AllowPrivilegeEscalation != nil && *c.SecurityContext.AllowPrivilegeEscalation {
41+
errs = append(errs, fmt.Errorf("container %q must NOT set AllowPrivilegeEscalation to true in its SecurityContext", c.Name))
42+
}
43+
if c.SecurityContext.Privileged != nil && *c.SecurityContext.Privileged {
44+
errs = append(errs, fmt.Errorf("container %q must NOT set Privileged to true in its SecurityContext", c.Name))
45+
}
46+
}
47+
return errors.NewAggregate(errs)
48+
}
49+
```
50+
51+
Check out [What are admission webhooks?](/docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-admission-webhooks)
52+
Or, see the [full code](webhook.go) of this webhook to follow along with this walkthrough.
53+
54+
## The policy
55+
Now let's try to recreate the validation faithfully with a ValidatingAdmissionPolicy.
56+
```yaml
57+
apiVersion: admissionregistration.k8s.io/v1
58+
kind: ValidatingAdmissionPolicy
59+
metadata:
60+
name: "pod-security.policy.example.com"
61+
spec:
62+
failurePolicy: Fail
63+
matchConstraints:
64+
resourceRules:
65+
- apiGroups: ["apps"]
66+
apiVersions: ["v1"]
67+
operations: ["CREATE", "UPDATE"]
68+
resources: ["deployments"]
69+
validations:
70+
- expression: object.spec.template.spec.containers.all(c, has(c.securityContext) && has(c.securityContext.runAsNonRoot) && c.securityContext.runAsNonRoot)
71+
message: 'all containers must set runAsNonRoot to true'
72+
- expression: object.spec.template.spec.containers.all(c, has(c.securityContext) && has(c.securityContext.readOnlyRootFilesystem) && c.securityContext.readOnlyRootFilesystem)
73+
message: 'all containers must set readOnlyRootFilesystem to true'
74+
- expression: object.spec.template.spec.containers.all(c, !has(c.securityContext) || !has(c.securityContext.allowPrivilegeEscalation) || !c.securityContext.allowPrivilegeEscalation)
75+
message: 'all containers must NOT set allowPrivilegeEscalation to true'
76+
- expression: object.spec.template.spec.containers.all(c, !has(c.securityContext) || !has(c.securityContext.Privileged) || !c.securityContext.Privileged)
77+
message: 'all containers must NOT set privileged to true'
78+
```
79+
Create the policy with `kubectl`. Great, no complain so far. But let's get the policy object back and take a look at its status.
80+
```shell
81+
kubectl get -oyaml validatingadmissionpolicies/pod-security.policy.example.com
82+
```
83+
```yaml
84+
status:
85+
typeChecking:
86+
expressionWarnings:
87+
- fieldRef: spec.validations[3].expression
88+
warning: |
89+
apps/v1, Kind=Deployment: ERROR: <input>:1:76: undefined field 'Privileged'
90+
| object.spec.template.spec.containers.all(c, !has(c.securityContext) || !has(c.securityContext.Privileged) || !c.securityContext.Privileged)
91+
| ...........................................................................^
92+
ERROR: <input>:1:128: undefined field 'Privileged'
93+
| object.spec.template.spec.containers.all(c, !has(c.securityContext) || !has(c.securityContext.Privileged) || !c.securityContext.Privileged)
94+
| ...............................................................................................................................^
95+
96+
```
97+
The policy was checked against its matched type, which is `apps/v1.Deployment`.
98+
Looking at the `fieldRef`, the problem was with the 3rd expression (index starts with 0)
99+
The expression in question accessed an undefined `Privileged` field.
100+
Ahh, looks like it was a copy-and-paste error. The field name should be in lowercase.
101+
102+
```yaml
103+
apiVersion: admissionregistration.k8s.io/v1
104+
kind: ValidatingAdmissionPolicy
105+
metadata:
106+
name: "pod-security.policy.example.com"
107+
spec:
108+
failurePolicy: Fail
109+
matchConstraints:
110+
resourceRules:
111+
- apiGroups: ["apps"]
112+
apiVersions: ["v1"]
113+
operations: ["CREATE", "UPDATE"]
114+
resources: ["deployments"]
115+
validations:
116+
- expression: object.spec.template.spec.containers.all(c, has(c.securityContext) && has(c.securityContext.runAsNonRoot) && c.securityContext.runAsNonRoot)
117+
message: 'all containers must set runAsNonRoot to true'
118+
- expression: object.spec.template.spec.containers.all(c, has(c.securityContext) && has(c.securityContext.readOnlyRootFilesystem) && c.securityContext.readOnlyRootFilesystem)
119+
message: 'all containers must set readOnlyRootFilesystem to true'
120+
- expression: object.spec.template.spec.containers.all(c, !has(c.securityContext) || !has(c.securityContext.allowPrivilegeEscalation) || !c.securityContext.allowPrivilegeEscalation)
121+
message: 'all containers must NOT set allowPrivilegeEscalation to true'
122+
- expression: object.spec.template.spec.containers.all(c, !has(c.securityContext) || !has(c.securityContext.privileged) || !c.securityContext.privileged)
123+
message: 'all containers must NOT set privileged to true'
124+
```
125+
Check its status again, and you should see all warnings cleared.
126+
127+
Next, let's create a namespace for our tests.
128+
```shell
129+
kubectl create namespace policy-test
130+
```
131+
Then, I bind the policy to the namespace. But at this point, I set the action to `Warn`
132+
so that the policy prints out [warnings](/blog/2020/09/03/warnings/) instead of rejecting the requests.
133+
This is especially useful to collect results from all expressions during development and automated testing.
134+
```yaml
135+
apiVersion: admissionregistration.k8s.io/v1
136+
kind: ValidatingAdmissionPolicyBinding
137+
metadata:
138+
name: "pod-security.policy-binding.example.com"
139+
spec:
140+
policyName: "pod-security.policy.example.com"
141+
validationActions: ["Warn"]
142+
matchResources:
143+
namespaceSelector:
144+
matchLabels:
145+
"kubernetes.io/metadata.name": "policy-test"
146+
```
147+
Tests out policy enforcement.
148+
```shell
149+
kubectl create -n policy-test -f- <<EOF
150+
apiVersion: apps/v1
151+
kind: Deployment
152+
metadata:
153+
labels:
154+
app: nginx
155+
name: nginx
156+
spec:
157+
selector:
158+
matchLabels:
159+
app: nginx
160+
template:
161+
metadata:
162+
labels:
163+
app: nginx
164+
spec:
165+
containers:
166+
- image: nginx
167+
name: nginx
168+
securityContext:
169+
privileged: true
170+
allowPrivilegeEscalation: true
171+
EOF
172+
```
173+
```text
174+
Warning: Validation failed for ValidatingAdmissionPolicy 'pod-security.policy.example.com' with binding 'pod-security.policy-binding.example.com': all containers must set runAsNonRoot to true
175+
Warning: Validation failed for ValidatingAdmissionPolicy 'pod-security.policy.example.com' with binding 'pod-security.policy-binding.example.com': all containers must set readOnlyRootFilesystem to true
176+
Warning: Validation failed for ValidatingAdmissionPolicy 'pod-security.policy.example.com' with binding 'pod-security.policy-binding.example.com': all containers must NOT set allowPrivilegeEscalation to true
177+
Warning: Validation failed for ValidatingAdmissionPolicy 'pod-security.policy.example.com' with binding 'pod-security.policy-binding.example.com': all containers must NOT set privileged to true
178+
Error from server: error when creating "STDIN": admission webhook "webhook.example.com" denied the request: [container "nginx" must set RunAsNonRoot to true in its SecurityContext, container "nginx" must set ReadOnlyRootFilesystem to true in its SecurityContext, container "nginx" must NOT set AllowPrivilegeEscalation to true in its SecurityContext, container "nginx" must NOT set Privileged to true in its SecurityContext]
179+
```
180+
Looks great! The policy and the webhook give equivalent results.
181+
After a few other cases, when we are confident with our policy, maybe it is time to do some cleanup.
182+
183+
- For every expression, we repeat access to `object.spec.template.spec.containers` and to each `securityContext`;
184+
- There is a pattern of checking presence of a field and then accessing it, which looks a bit verbose.
185+
186+
Fortunately, since Kubernetes 1.28, we have new solutions for both issues.
187+
Variable Composition allows us to extract repeated sub-expressions into their own variables.
188+
Kubernetes enables [the optional library](https://github.com/google/cel-spec/wiki/proposal-246) for CEL, which are excellent to work with fields that are, you guessed it, optional.
189+
190+
With both features in mind, let's refactor the policy a bit.
191+
```yaml
192+
apiVersion: admissionregistration.k8s.io/v1
193+
kind: ValidatingAdmissionPolicy
194+
metadata:
195+
name: "pod-security.policy.example.com"
196+
spec:
197+
failurePolicy: Fail
198+
matchConstraints:
199+
resourceRules:
200+
- apiGroups: ["apps"]
201+
apiVersions: ["v1"]
202+
operations: ["CREATE", "UPDATE"]
203+
resources: ["deployments"]
204+
variables:
205+
- name: containers
206+
expression: object.spec.template.spec.containers
207+
- name: securityContexts
208+
expression: 'variables.containers.map(c, c.?securityContext)'
209+
validations:
210+
- expression: variables.securityContexts.all(c, c.?runAsNonRoot == optional.of(true))
211+
message: 'all containers must set runAsNonRoot to true'
212+
- expression: variables.securityContexts.all(c, c.?readOnlyRootFilesystem == optional.of(true))
213+
message: 'all containers must set readOnlyRootFilesystem to true'
214+
- expression: variables.securityContexts.all(c, c.?allowPrivilegeEscalation != optional.of(true))
215+
message: 'all containers must NOT set allowPrivilegeEscalation to true'
216+
- expression: variables.securityContexts.all(c, c.?privileged != optional.of(true))
217+
message: 'all containers must NOT set privileged to true'
218+
```
219+
The policy is now much cleaner and more readable. Update the policy, and you should see
220+
it function the same as before.
221+
222+
Now let's change the policy binding from warning to actually denying requests that fail validation.
223+
```yaml
224+
apiVersion: admissionregistration.k8s.io/v1
225+
kind: ValidatingAdmissionPolicyBinding
226+
metadata:
227+
name: "pod-security.policy-binding.example.com"
228+
spec:
229+
policyName: "pod-security.policy.example.com"
230+
validationActions: ["Deny"]
231+
matchResources:
232+
namespaceSelector:
233+
matchLabels:
234+
"kubernetes.io/metadata.name": "policy-test"
235+
```
236+
And finally, remove the webhook. Now the result should include only messages from
237+
the policy.
238+
```shell
239+
kubectl create -n policy-test -f- <<EOF
240+
apiVersion: apps/v1
241+
kind: Deployment
242+
metadata:
243+
labels:
244+
app: nginx
245+
name: nginx
246+
spec:
247+
selector:
248+
matchLabels:
249+
app: nginx
250+
template:
251+
metadata:
252+
labels:
253+
app: nginx
254+
spec:
255+
containers:
256+
- image: nginx
257+
name: nginx
258+
securityContext:
259+
privileged: true
260+
allowPrivilegeEscalation: true
261+
EOF
262+
```
263+
```text
264+
The deployments "nginx" is invalid: : ValidatingAdmissionPolicy 'pod-security.policy.example.com' with binding 'pod-security.policy-binding.example.com' denied request: all containers must set runAsNonRoot to true
265+
```
266+
Please notice that, by design, the policy will stop evaluation after the first expression that causes the request to be denied.
267+
This is different from what happens when the expressions generate only warnings.
268+
269+
## Set up monitoring
270+
Unlike a webhook, a policy is not a dedicated process that can expose its own metrics.
271+
Instead, you can use metrics from the API server in their place.
272+
273+
Here are some examples in Prometheus Query Language of common monitoring tasks.
274+
275+
To find the 95th percentile execution duration of the policy shown above.
276+
```text
277+
histogram_quantile(0.95, sum(rate(apiserver_validating_admission_policy_check_duration_seconds_bucket{policy="pod-security.policy.example.com"}[5m])) by (le))
278+
```
279+
280+
To find the rate of the policy evaluation.
281+
```text
282+
rate(apiserver_validating_admission_policy_check_total{policy="pod-security.policy.example.com"}[5m])
283+
```
284+
285+
You can read [the metrics reference](/docs/reference/instrumentation/metrics/) to learn more about the metrics above.
286+
The metrics of ValidatingAdmissionPolicy are currently in alpha,
287+
and more and better metrics will come while the stability graduates in the future release.

‎content/en/blog/_posts/2024-04-24-validating-admission-policy-ga/webhook.go

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
package main
2+
3+
import (
4+
"encoding/json"
5+
"flag"
6+
"fmt"
7+
"log"
8+
"net/http"
9+
10+
admissionv1 "k8s.io/api/admission/v1"
11+
appsv1 "k8s.io/api/apps/v1"
12+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
13+
"k8s.io/apimachinery/pkg/util/errors"
14+
)
15+
16+
func verifyDeployment(deploy *appsv1.Deployment) error {
17+
var errs []error
18+
for i, c := range deploy.Spec.Template.Spec.Containers {
19+
if c.Name == "" {
20+
return fmt.Errorf("container %d has no name", i)
21+
}
22+
if c.SecurityContext == nil {
23+
errs = append(errs, fmt.Errorf("container %q does not have SecurityContext", c.Name))
24+
}
25+
if c.SecurityContext.RunAsNonRoot == nil || !*c.SecurityContext.RunAsNonRoot {
26+
errs = append(errs, fmt.Errorf("container %q must set RunAsNonRoot to true in its SecurityContext", c.Name))
27+
}
28+
if c.SecurityContext.ReadOnlyRootFilesystem == nil || !*c.SecurityContext.ReadOnlyRootFilesystem {
29+
errs = append(errs, fmt.Errorf("container %q must set ReadOnlyRootFilesystem to true in its SecurityContext", c.Name))
30+
}
31+
if c.SecurityContext.AllowPrivilegeEscalation != nil && *c.SecurityContext.AllowPrivilegeEscalation {
32+
errs = append(errs, fmt.Errorf("container %q must NOT set AllowPrivilegeEscalation to true in its SecurityContext", c.Name))
33+
}
34+
if c.SecurityContext.Privileged != nil && *c.SecurityContext.Privileged {
35+
errs = append(errs, fmt.Errorf("container %q must NOT set Privileged to true in its SecurityContext", c.Name))
36+
}
37+
}
38+
return errors.NewAggregate(errs)
39+
}
40+
41+
func WebhookEnforceSecurePodConfiguration(rw http.ResponseWriter, req *http.Request) {
42+
result := &admissionv1.AdmissionReview{Response: &admissionv1.AdmissionResponse{}}
43+
err := func() error {
44+
ar := new(admissionv1.AdmissionReview)
45+
err := json.NewDecoder(req.Body).Decode(ar)
46+
if err != nil {
47+
return err
48+
}
49+
if ar.Request == nil {
50+
return nil
51+
}
52+
result.TypeMeta = ar.TypeMeta
53+
result.Response.UID = ar.Request.UID
54+
if len(ar.Request.Object.Raw) == 0 {
55+
return nil
56+
}
57+
deploy := new(appsv1.Deployment)
58+
err = json.Unmarshal(ar.Request.Object.Raw, deploy)
59+
if err != nil {
60+
return err
61+
}
62+
return verifyDeployment(deploy)
63+
}()
64+
if err == nil {
65+
result.Response.Allowed = true
66+
} else {
67+
result.Response.Allowed = false
68+
result.Response.Result = &metav1.Status{
69+
Code: http.StatusForbidden,
70+
Message: err.Error(),
71+
}
72+
}
73+
err = json.NewEncoder(rw).Encode(result)
74+
if err != nil {
75+
log.Println(err)
76+
}
77+
}
78+
79+
var _ http.HandlerFunc = WebhookEnforceSecurePodConfiguration
80+
81+
func main() {
82+
http.HandleFunc("/", WebhookEnforceSecurePodConfiguration)
83+
84+
addr := flag.String("addr", ":8443", "address to listen on")
85+
certFile := flag.String("cert", "cert.pem", "path to TLS certificate")
86+
keyFile := flag.String("key", "key.pem", "path to TLS key")
87+
flag.Parse()
88+
89+
log.Fatalln(http.ListenAndServeTLS(*addr, *certFile, *keyFile, nil))
90+
}

0 commit comments

Comments
 (0)