OCP BUGS-1585: Adding how to find workloads causing pod security violations

Author: bergerhoffer

authentication/understanding-and-managing-pod-security-admission.adoc

@@ -8,12 +8,18 @@ toc::[]
 
 Pod security admission is an implementation of the link:https://kubernetes.io/docs/concepts/security/pod-security-standards/[Kubernetes pod security standards]. Use pod security admission to restrict the behavior of pods.
 
+// Security context constraint synchronization with pod security standards
 include::modules/security-context-constraints-psa-synchronization.adoc[leveloffset=+1]
 
+// Controlling pod security admission synchronization
 include::modules/security-context-constraints-psa-opting.adoc[leveloffset=+1]
 
+// About pod security admission alerts
 include::modules/security-context-constraints-psa-rectifying.adoc[leveloffset=+1]
 
+// Identifying pod security violations
+include::modules/security-context-constraints-psa-alert-eval.adoc[leveloffset=+2]
+
 [role="_additional-resources"]
 [id="additional-resources_managing-pod-security-admission"]
 == Additional resources

modules/security-context-constraints-psa-alert-eval.adoc

@@ -0,0 +1,42 @@
+// Module included in the following assemblies:
+//
+// * authentication/understanding-and-managing-pod-security-admission.adoc
+
+:_content-type: PROCEDURE
+[id="security-context-constraints-psa-alert-eval_{context}"]
+= Identifying pod security violations
+
+The `PodSecurityViolation` alert does not provide details on which workloads are causing pod security violations. You can identify the affected workloads by reviewing the Kubernetes API server audit logs. This procedure uses the `must-gather` tool to gather the audit logs and then searches for the `pod-security.kubernetes.io/audit-violations` annotation.
+
+.Prerequisites
+
+* You have installed `jq`.
+* You have access to the cluster as a user with the `cluster-admin` role.
+
+.Procedure
+
+. To gather the audit logs, enter the following command:
++
+[source,terminal]
+----
+$ oc adm must-gather -- /usr/bin/gather_audit_logs
+----
+
+. To output the affected workload details, enter the following command:
++
+[source,terminal]
+----
+$ zgrep -h pod-security.kubernetes.io/audit-violations must-gather.local.<archive_id>/quay*/audit_logs/kube-apiserver/*log.gz \
+  | jq -r 'select((.annotations["pod-security.kubernetes.io/audit-violations"] != null) and (.objectRef.resource=="pods")) | .objectRef.namespace + " " + .objectRef.name + " " + .objectRef.resource' \
+  | sort | uniq -c
+----
++
+Replace `must-gather.local.<archive_id>` with the actual directory name.
++
+.Example output
+[source,text]
+----
+15 ci namespace-ttl-controller deployments
+ 1 ci-op-k5whzrsh rpm-repo-546f98d8b replicasets
+ 1 ci-op-k5whzrsh rpm-repo deployments
+----

modules/security-context-constraints-psa-rectifying.adoc

@@ -2,7 +2,7 @@
 //
 // * authentication/understanding-and-managing-pod-security-admission.adoc
 
-:_content-type: PROCEDURE
+:_content-type: CONCEPT
 [id="security-context-constraints-psa-rectifying_{context}"]
 = About pod security admission alerts