Skip to content

Commit d29e93a

Browse files
PushkarJPushkarJ
committed
Tutorial for pod security admission
Refer blog post for v1.23 + suggestions from code review
1 parent e808eda commit d29e93a

File tree

4 files changed

+489
-0
lines changed

4 files changed

+489
-0
lines changed

‎content/en/docs/tutorials/_index.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,11 @@ Before walking through each tutorial, you may want to bookmark the
5757

5858
* [Using Source IP](/docs/tutorials/services/source-ip/)
5959

60+
## Security
61+
62+
* [Applying Pod Security Standards at Cluster level](/docs/tutorials/security/cluster-level-pss/)
63+
* [Applying Pod Security Standards at Namespace level](/docs/tutorials/security/ns-level-pss/)
64+
6065
## {{% heading "whatsnext" %}}
6166

6267
If you would like to write a tutorial, see

‎content/en/docs/tutorials/security/_index.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
title: "Security"
3+
weight: 40
4+
---
5+

‎content/en/docs/tutorials/security/cluster-level-pss.md

Lines changed: 319 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,319 @@
1+
---
2+
title: Applying Pod Security Standards at the cluster level
3+
content_type: tutorial
4+
weight: 10
5+
---
6+
7+
{{% alert title="Note" %}}
8+
This tutorial applies only for new clusters.
9+
{{% /alert %}}
10+
11+
Pod Security admission (PSA) is enabled by default in v1.23 and later, as it [graduated
12+
to beta](/blog/2021/12/15/pod-security-admission-beta/). Pod Security Admission
13+
is an admission controller that applies Pod Security Standards when pods are
14+
created. This tutorial shows you how to enforce the `baseline` Pod Security
15+
Standard at the cluster level which applies a standard configuration
16+
to all namespaces in a cluster.
17+
18+
For applying pod security standards one namespace at a time, please [follow this
19+
tutorial](/docs/tutorials/security/ns-level-pss).
20+
21+
## {{% heading "prerequisites" %}}
22+
23+
Install the following on your workstation:
24+
25+
- [KinD](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
26+
- [kubectl](https://kubernetes.io/docs/tasks/tools/)
27+
28+
## Choose the right Pod Security Standard to apply
29+
30+
[Pod Security Admission](/docs/concepts/security/pod-security-admission/)
31+
lets you apply built-in [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
32+
with the following modes: `enforce`, `audit`, and `warn`.
33+
34+
To gather information that helps you to choose the Pod Security Standards
35+
that are most appropriate for your configuration, do the following:
36+
37+
1. Create a cluster with no Pod Security Standards applied:
38+
39+
```shell
40+
kind create cluster --name psa-wo-cluster-pss --image kindest/node:latest
41+
```
42+
The output is similar to this:
43+
```
44+
Creating cluster "psa-wo-cluster-pss" ...
45+
✓ Ensuring node image (kindest/node:latest) 🖼
46+
✓ Preparing nodes 📦
47+
✓ Writing configuration 📜
48+
✓ Starting control-plane 🕹️
49+
✓ Installing CNI 🔌
50+
✓ Installing StorageClass 💾
51+
Set kubectl context to "kind-psa-wo-cluster-pss"
52+
You can now use your cluster with:
53+
54+
kubectl cluster-info --context kind-psa-wo-cluster-pss
55+
56+
Thanks for using kind! 😊
57+
58+
```
59+
60+
2. Set the kubectl context to the new cluster:
61+
62+
```shell
63+
kubectl cluster-info --context kind-psa-wo-cluster-pss
64+
```
65+
The output is similar to this:
66+
67+
```
68+
Kubernetes control plane is running at https://127.0.0.1:61350
69+
70+
CoreDNS is running at https://127.0.0.1:61350/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
71+
72+
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
73+
```
74+
75+
3. Get a list of namespaces in the cluster:
76+
77+
```shell
78+
kubectl get ns
79+
```
80+
The output is similar to this:
81+
```
82+
NAME STATUS AGE
83+
default Active 9m30s
84+
kube-node-lease Active 9m32s
85+
kube-public Active 9m32s
86+
kube-system Active 9m32s
87+
local-path-storage Active 9m26s
88+
```
89+
90+
4. Use `--dry-run=server` to understand what happens when different Pod Security Standards
91+
are applied:
92+
93+
1. Privileged
94+
```shell
95+
kubectl label --dry-run=server --overwrite ns --all \
96+
pod-security.kubernetes.io/enforce=privileged
97+
```
98+
The output is similar to this:
99+
```
100+
namespace/default labeled
101+
namespace/kube-node-lease labeled
102+
namespace/kube-public labeled
103+
namespace/kube-system labeled
104+
namespace/local-path-storage labeled
105+
```
106+
2. Baseline
107+
```shell
108+
kubectl label --dry-run=server --overwrite ns --all \
109+
pod-security.kubernetes.io/enforce=baseline
110+
```
111+
The output is similar to this:
112+
```
113+
namespace/default labeled
114+
namespace/kube-node-lease labeled
115+
namespace/kube-public labeled
116+
Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "baseline:latest"
117+
Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes
118+
Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes
119+
Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged
120+
namespace/kube-system labeled
121+
namespace/local-path-storage labeled
122+
```
123+
124+
3. Restricted
125+
```shell
126+
kubectl label --dry-run=server --overwrite ns --all \
127+
pod-security.kubernetes.io/enforce=restricted
128+
```
129+
The output is similar to this:
130+
```
131+
namespace/default labeled
132+
namespace/kube-node-lease labeled
133+
namespace/kube-public labeled
134+
Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "restricted:latest"
135+
Warning: coredns-7bb9c7b568-hsptc (and 1 other pod): unrestricted capabilities, runAsNonRoot != true, seccompProfile
136+
Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true
137+
Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
138+
Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
139+
namespace/kube-system labeled
140+
Warning: existing pods in namespace "local-path-storage" violate the new PodSecurity enforce level "restricted:latest"
141+
Warning: local-path-provisioner-d6d9f7ffc-lw9lh: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
142+
namespace/local-path-storage labeled
143+
```
144+
145+
From the previous output, you'll notice that applying the `privileged` Pod Security Standard shows no warnings
146+
for any namespaces. However, `baseline` and `restricted` standards both have
147+
warnings, specifically in the `kube-system` namespace.
148+
149+
## Set modes, versions and standards
150+
151+
In this tutorial, you apply the following Pod Security Standards to the `latest` version:
152+
153+
* `baseline` standard in `enforce` mode.
154+
* `restricted` standard in `warn` and `audit` mode.
155+
156+
The `baseline` Pod Security Standard provides a convenient
157+
middle ground that allows keeping the exemption list short and prevents known
158+
privilege escalations.
159+
160+
Additionally, to prevent pods from failing in `kube-system`, you'll exempt the namespace
161+
from having Pod Security Standards applied.
162+
163+
When you implement Pod Security Admission in your own environment, consider the
164+
following:
165+
166+
1. Based on the risk posture applied to a cluster, a stricter Pod Security
167+
Standard like `restricted` might be a better choice.
168+
1. Exempting the `kube-system` namespace allows pods to run as
169+
`privileged` in this namespace. We recommend that you apply strict RBAC
170+
policies that limit access to `kube-system`, following the principle of least
171+
privilege.
172+
173+
1. Create a configuration file that can be consumed by the Pod Security
174+
Admission Controller to implement these Pod Security Standards:
175+
176+
```
177+
mkdir -p /tmp/pss
178+
cat <<EOF > /tmp/pss/cluster-level-pss.yaml
179+
apiVersion: apiserver.config.k8s.io/v1
180+
kind: AdmissionConfiguration
181+
plugins:
182+
- name: PodSecurity
183+
configuration:
184+
apiVersion: pod-security.admission.config.k8s.io/v1beta1
185+
kind: PodSecurityConfiguration
186+
defaults:
187+
enforce: "baseline"
188+
enforce-version: "latest"
189+
audit: "restricted"
190+
audit-version: "latest"
191+
warn: "restricted"
192+
warn-version: "latest"
193+
exemptions:
194+
usernames: []
195+
runtimeClasses: []
196+
namespaces: [kube-system]
197+
EOF
198+
```
199+
200+
201+
1. Configure the API server to consume this file during cluster creation:
202+
203+
```
204+
cat <<EOF > /tmp/pss/cluster-config.yaml
205+
kind: Cluster
206+
apiVersion: kind.x-k8s.io/v1alpha4
207+
nodes:
208+
- role: control-plane
209+
kubeadmConfigPatches:
210+
- |
211+
kind: ClusterConfiguration
212+
apiServer:
213+
extraArgs:
214+
admission-control-config-file: /etc/config/cluster-level-pss.yaml
215+
extraVolumes:
216+
- name: accf
217+
hostPath: /etc/config
218+
mountPath: /etc/config
219+
readOnly: false
220+
pathType: "DirectoryOrCreate"
221+
extraMounts:
222+
- hostPath: /tmp/pss
223+
containerPath: /etc/config
224+
# optional: if set, the mount is read-only.
225+
# default false
226+
readOnly: false
227+
# optional: if set, the mount needs SELinux relabeling.
228+
# default false
229+
selinuxRelabel: false
230+
# optional: set propagation mode (None, HostToContainer or Bidirectional)
231+
# see https://kubernetes.io/docs/concepts/storage/volumes/#mount-propagation
232+
# default None
233+
propagation: None
234+
EOF
235+
```
236+
237+
{{<note>}}
238+
If you use Docker Desktop with KinD, the `/tmp`
239+
directory is added as a Shared Directory under
240+
**Preferences > Resources > File Sharing** on Mac OS.
241+
{{</note>}}
242+
243+
2. Create a cluster that uses Pod Security Admission to apply
244+
these Pod Security Standards:
245+
246+
```shell
247+
kind create cluster --name psa-with-cluster-pss --image kindest/node:latest --config /tmp/pss/cluster-config.yaml
248+
```
249+
The output is similar to this:
250+
```
251+
Creating cluster "psa-with-cluster-pss" ...
252+
✓ Ensuring node image (kindest/node:latest) 🖼
253+
✓ Preparing nodes 📦
254+
✓ Writing configuration 📜
255+
✓ Starting control-plane 🕹️
256+
✓ Installing CNI 🔌
257+
✓ Installing StorageClass 💾
258+
Set kubectl context to "kind-psa-with-cluster-pss"
259+
You can now use your cluster with:
260+
261+
kubectl cluster-info --context kind-psa-with-cluster-pss
262+
263+
Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
264+
265+
3. Point kubectl to the cluster
266+
```shell
267+
kubectl cluster-info --context kind-psa-with-cluster-pss
268+
Kubernetes control plane is running at https://127.0.0.1:63855
269+
CoreDNS is running at https://127.0.0.1:63855/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
270+
271+
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
272+
```
273+
274+
4. Create a Pod with minimal configuration in the default namespace:
275+
276+
```
277+
cat <<EOF > /tmp/pss/nginx-pod.yaml
278+
apiVersion: v1
279+
kind: Pod
280+
metadata:
281+
name: nginx
282+
spec:
283+
containers:
284+
- image: nginx
285+
name: nginx
286+
ports:
287+
- containerPort: 80
288+
EOF
289+
```
290+
5. Create Pod after pod security is enabled at cluster level:
291+
292+
```shell
293+
kubectl apply -f /tmp/pss/nginx-pod.yaml
294+
```
295+
The output is similar to this:
296+
```
297+
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
298+
pod/nginx created
299+
```
300+
## Clean up
301+
302+
Run `kind delete cluster -name psa-with-cluster-pss` and
303+
`kind delete cluster -name psa-wo-cluster-pss` to delete the clusters you
304+
created.
305+
306+
## {{% heading "whatsnext" %}}
307+
308+
- Run a
309+
[gist](https://gist.github.com/PushkarJ/9f7a0045f4bec31097bdd1e9db0f2f6e)
310+
to perform all the preceding steps at once:
311+
1. Create a Pod Security Standards based cluster level Configuration
312+
2. Create a file to let API server consumes this configuration
313+
3. Create a cluster that creates an API server with this configuration
314+
4. Set kubectl context to this new cluster
315+
5. Create a minimal pod yaml file
316+
6. Apply this file to create a Pod in the new cluster
317+
- [Pod Security Admission](/docs/concepts/security/pod-security-admission/)
318+
- [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
319+
- [Applying Pod Security Standards at the namespace level](/docs/tutorials/security/ns-level-pss/)

0 commit comments

Comments
 (0)