Skip to content

Commit aa848ab

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #30422 from PushkarJ/psa-tutorial
Tutorials for Pod Security Admission
2 parents 2740d85 + d1e2545 commit aa848ab

File tree

6 files changed

+591
-0
lines changed

6 files changed

+591
-0
lines changed

‎content/en/docs/tutorials/_index.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,11 @@ Before walking through each tutorial, you may want to bookmark the
5757

5858
* [Using Source IP](/docs/tutorials/services/source-ip/)
5959

60+
## Security
61+
62+
* [Apply Pod Security Standards at Cluster level](/docs/tutorials/security/cluster-level-pss/)
63+
* [Apply Pod Security Standards at Namespace level](/docs/tutorials/security/ns-level-pss/)
64+
6065
## {{% heading "whatsnext" %}}
6166

6267
If you would like to write a tutorial, see

‎content/en/docs/tutorials/security/_index.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
title: "Security"
3+
weight: 40
4+
---
5+

‎content/en/docs/tutorials/security/cluster-level-pss.md

Lines changed: 324 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,324 @@
1+
---
2+
title: Apply Pod Security Standards at the Cluster Level
3+
content_type: tutorial
4+
weight: 10
5+
---
6+
7+
{{% alert title="Note" %}}
8+
This tutorial applies only for new clusters.
9+
{{% /alert %}}
10+
11+
Pod Security admission (PSA) is enabled by default in v1.23 and later, as it has
12+
[graduated to beta](/blog/2021/12/09/pod-security-admission-beta/).
13+
Pod Security
14+
is an admission controller that carries out checks against the Kubernetes
15+
[Pod Security Standards](docs/concepts/security/pod-security-standards/) when new pods are
16+
created. This tutorial shows you how to enforce the `baseline` Pod Security
17+
Standard at the cluster level which applies a standard configuration
18+
to all namespaces in a cluster.
19+
20+
To apply Pod Security Standards to specific namespaces, refer to [Apply Pod Security Standards at the namespace level](/docs/tutorials/security/ns-level-pss).
21+
22+
## {{% heading "prerequisites" %}}
23+
24+
Install the following on your workstation:
25+
26+
- [KinD](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
27+
- [kubectl](https://kubernetes.io/docs/tasks/tools/)
28+
29+
## Choose the right Pod Security Standard to apply
30+
31+
[Pod Security Admission](/docs/concepts/security/pod-security-admission/)
32+
lets you apply built-in [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
33+
with the following modes: `enforce`, `audit`, and `warn`.
34+
35+
To gather information that helps you to choose the Pod Security Standards
36+
that are most appropriate for your configuration, do the following:
37+
38+
1. Create a cluster with no Pod Security Standards applied:
39+
40+
```shell
41+
kind create cluster --name psa-wo-cluster-pss --image kindest/node:v1.23.0
42+
```
43+
The output is similar to this:
44+
```
45+
Creating cluster "psa-wo-cluster-pss" ...
46+
✓ Ensuring node image (kindest/node:v1.23.0) 🖼
47+
✓ Preparing nodes 📦
48+
✓ Writing configuration 📜
49+
✓ Starting control-plane 🕹️
50+
✓ Installing CNI 🔌
51+
✓ Installing StorageClass 💾
52+
Set kubectl context to "kind-psa-wo-cluster-pss"
53+
You can now use your cluster with:
54+
55+
kubectl cluster-info --context kind-psa-wo-cluster-pss
56+
57+
Thanks for using kind! 😊
58+
59+
```
60+
61+
1. Set the kubectl context to the new cluster:
62+
63+
```shell
64+
kubectl cluster-info --context kind-psa-wo-cluster-pss
65+
```
66+
The output is similar to this:
67+
68+
```
69+
Kubernetes control plane is running at https://127.0.0.1:61350
70+
71+
CoreDNS is running at https://127.0.0.1:61350/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
72+
73+
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
74+
```
75+
76+
1. Get a list of namespaces in the cluster:
77+
78+
```shell
79+
kubectl get ns
80+
```
81+
The output is similar to this:
82+
```
83+
NAME STATUS AGE
84+
default Active 9m30s
85+
kube-node-lease Active 9m32s
86+
kube-public Active 9m32s
87+
kube-system Active 9m32s
88+
local-path-storage Active 9m26s
89+
```
90+
91+
1. Use `--dry-run=server` to understand what happens when different Pod Security Standards
92+
are applied:
93+
94+
1. Privileged
95+
```shell
96+
kubectl label --dry-run=server --overwrite ns --all \
97+
pod-security.kubernetes.io/enforce=privileged
98+
```
99+
The output is similar to this:
100+
```
101+
namespace/default labeled
102+
namespace/kube-node-lease labeled
103+
namespace/kube-public labeled
104+
namespace/kube-system labeled
105+
namespace/local-path-storage labeled
106+
```
107+
2. Baseline
108+
```shell
109+
kubectl label --dry-run=server --overwrite ns --all \
110+
pod-security.kubernetes.io/enforce=baseline
111+
```
112+
The output is similar to this:
113+
```
114+
namespace/default labeled
115+
namespace/kube-node-lease labeled
116+
namespace/kube-public labeled
117+
Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "baseline:latest"
118+
Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes
119+
Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes
120+
Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged
121+
namespace/kube-system labeled
122+
namespace/local-path-storage labeled
123+
```
124+
125+
3. Restricted
126+
```shell
127+
kubectl label --dry-run=server --overwrite ns --all \
128+
pod-security.kubernetes.io/enforce=restricted
129+
```
130+
The output is similar to this:
131+
```
132+
namespace/default labeled
133+
namespace/kube-node-lease labeled
134+
namespace/kube-public labeled
135+
Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "restricted:latest"
136+
Warning: coredns-7bb9c7b568-hsptc (and 1 other pod): unrestricted capabilities, runAsNonRoot != true, seccompProfile
137+
Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true
138+
Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
139+
Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
140+
namespace/kube-system labeled
141+
Warning: existing pods in namespace "local-path-storage" violate the new PodSecurity enforce level "restricted:latest"
142+
Warning: local-path-provisioner-d6d9f7ffc-lw9lh: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
143+
namespace/local-path-storage labeled
144+
```
145+
146+
From the previous output, you'll notice that applying the `privileged` Pod Security Standard shows no warnings
147+
for any namespaces. However, `baseline` and `restricted` standards both have
148+
warnings, specifically in the `kube-system` namespace.
149+
150+
## Set modes, versions and standards
151+
152+
In this section, you apply the following Pod Security Standards to the `latest` version:
153+
154+
* `baseline` standard in `enforce` mode.
155+
* `restricted` standard in `warn` and `audit` mode.
156+
157+
The `baseline` Pod Security Standard provides a convenient
158+
middle ground that allows keeping the exemption list short and prevents known
159+
privilege escalations.
160+
161+
Additionally, to prevent pods from failing in `kube-system`, you'll exempt the namespace
162+
from having Pod Security Standards applied.
163+
164+
When you implement Pod Security Admission in your own environment, consider the
165+
following:
166+
167+
1. Based on the risk posture applied to a cluster, a stricter Pod Security
168+
Standard like `restricted` might be a better choice.
169+
1. Exempting the `kube-system` namespace allows pods to run as
170+
`privileged` in this namespace. For real world use, the Kubernetes project
171+
strongly recommends that you apply strict RBAC
172+
policies that limit access to `kube-system`, following the principle of least
173+
privilege.
174+
To implement the preceding standards, do the following:
175+
1. Create a configuration file that can be consumed by the Pod Security
176+
Admission Controller to implement these Pod Security Standards:
177+
178+
```
179+
mkdir -p /tmp/pss
180+
cat <<EOF > /tmp/pss/cluster-level-pss.yaml
181+
apiVersion: apiserver.config.k8s.io/v1
182+
kind: AdmissionConfiguration
183+
plugins:
184+
- name: PodSecurity
185+
configuration:
186+
apiVersion: pod-security.admission.config.k8s.io/v1beta1
187+
kind: PodSecurityConfiguration
188+
defaults:
189+
enforce: "baseline"
190+
enforce-version: "latest"
191+
audit: "restricted"
192+
audit-version: "latest"
193+
warn: "restricted"
194+
warn-version: "latest"
195+
exemptions:
196+
usernames: []
197+
runtimeClasses: []
198+
namespaces: [kube-system]
199+
EOF
200+
```
201+
202+
1. Configure the API server to consume this file during cluster creation:
203+
204+
```
205+
cat <<EOF > /tmp/pss/cluster-config.yaml
206+
kind: Cluster
207+
apiVersion: kind.x-k8s.io/v1alpha4
208+
nodes:
209+
- role: control-plane
210+
kubeadmConfigPatches:
211+
- |
212+
kind: ClusterConfiguration
213+
apiServer:
214+
extraArgs:
215+
admission-control-config-file: /etc/config/cluster-level-pss.yaml
216+
extraVolumes:
217+
- name: accf
218+
hostPath: /etc/config
219+
mountPath: /etc/config
220+
readOnly: false
221+
pathType: "DirectoryOrCreate"
222+
extraMounts:
223+
- hostPath: /tmp/pss
224+
containerPath: /etc/config
225+
# optional: if set, the mount is read-only.
226+
# default false
227+
readOnly: false
228+
# optional: if set, the mount needs SELinux relabeling.
229+
# default false
230+
selinuxRelabel: false
231+
# optional: set propagation mode (None, HostToContainer or Bidirectional)
232+
# see https://kubernetes.io/docs/concepts/storage/volumes/#mount-propagation
233+
# default None
234+
propagation: None
235+
EOF
236+
```
237+
238+
{{<note>}}
239+
If you use Docker Desktop with KinD on macOS, you can
240+
add `/tmp` as a Shared Directory under the menu item
241+
**Preferences > Resources > File Sharing**.
242+
{{</note>}}
243+
244+
1. Create a cluster that uses Pod Security Admission to apply
245+
these Pod Security Standards:
246+
247+
```shell
248+
kind create cluster --name psa-with-cluster-pss --image kindest/node:v1.23.0 --config /tmp/pss/cluster-config.yaml
249+
```
250+
The output is similar to this:
251+
```
252+
Creating cluster "psa-with-cluster-pss" ...
253+
✓ Ensuring node image (kindest/node:v1.23.0) 🖼
254+
✓ Preparing nodes 📦
255+
✓ Writing configuration 📜
256+
✓ Starting control-plane 🕹️
257+
✓ Installing CNI 🔌
258+
✓ Installing StorageClass 💾
259+
Set kubectl context to "kind-psa-with-cluster-pss"
260+
You can now use your cluster with:
261+
262+
kubectl cluster-info --context kind-psa-with-cluster-pss
263+
264+
Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
265+
```
266+
267+
1. Point kubectl to the cluster
268+
```shell
269+
kubectl cluster-info --context kind-psa-with-cluster-pss
270+
```
271+
The output is similar to this:
272+
```
273+
Kubernetes control plane is running at https://127.0.0.1:63855
274+
CoreDNS is running at https://127.0.0.1:63855/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
275+
276+
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
277+
```
278+
1. Create the following Pod specification for a minimal configuration in the default namespace:
279+
280+
```
281+
cat <<EOF > /tmp/pss/nginx-pod.yaml
282+
apiVersion: v1
283+
kind: Pod
284+
metadata:
285+
name: nginx
286+
spec:
287+
containers:
288+
- image: nginx
289+
name: nginx
290+
ports:
291+
- containerPort: 80
292+
EOF
293+
```
294+
1. Create the Pod in the cluster:
295+
296+
```shell
297+
kubectl apply -f /tmp/pss/nginx-pod.yaml
298+
```
299+
The output is similar to this:
300+
```
301+
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
302+
pod/nginx created
303+
```
304+
305+
## Clean up
306+
307+
Run `kind delete cluster -name psa-with-cluster-pss` and
308+
`kind delete cluster -name psa-wo-cluster-pss` to delete the clusters you
309+
created.
310+
311+
## {{% heading "whatsnext" %}}
312+
313+
- Run a
314+
[shell script](/examples/security/kind-with-cluster-level-baseline-pod-security.sh)
315+
to perform all the preceding steps at once:
316+
1. Create a Pod Security Standards based cluster level Configuration
317+
2. Create a file to let API server consumes this configuration
318+
3. Create a cluster that creates an API server with this configuration
319+
4. Set kubectl context to this new cluster
320+
5. Create a minimal pod yaml file
321+
6. Apply this file to create a Pod in the new cluster
322+
- [Pod Security Admission](/docs/concepts/security/pod-security-admission/)
323+
- [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
324+
- [Apply Pod Security Standards at the namespace level](/docs/tutorials/security/ns-level-pss/)

0 commit comments

Comments
 (0)