KEP-3619: Fine-grained SupplementalGroups control

Author: everpeace

content/en/docs/reference/command-line-tools-reference/feature-gates/supplemental-groups-policy.md

@@ -0,0 +1,14 @@
+---
+title: SupplementalGroupsPolicy
+content_type: feature_gate
+_build:
+  list: never
+  render: false
+
+stages:
+  - stage: alpha
+    defaultValue: false
+    fromVersion: "1.31"
+---
+Enables support for fine-grained SupplementalGroups control.
+For more details, see [Configure fine-grained SupplementalGroups control for a Pod](/content/en/docs/tasks/configure-pod-container/security-context/#supplementalgroupspolicy).

content/en/docs/tasks/configure-pod-container/security-context.md

@@ -66,6 +66,8 @@ all processes within any containers of the Pod. If this field is omitted, the pr
 will be root(0). Any files created will also be owned by user 1000 and group 3000 when `runAsGroup` is specified.
 Since `fsGroup` field is specified, all processes of the container are also part of the supplementary group ID 2000.
 The owner for volume `/data/demo` and any files created in that volume will be Group ID 2000.
+Additionally, since `supplementalGroups` field is specified, all processes of the container are also part of the
+specified group IDs. If this field is omitted, it means empty.
 
 Create the Pod:
 
@@ -142,20 +144,189 @@ id
 The output is similar to this:
 
 ```none
-uid=1000 gid=3000 groups=2000
+uid=1000 gid=3000 groups=2000,3000,4000
 ```
 
 From the output, you can see that `gid` is 3000 which is same as the `runAsGroup` field.
 If the `runAsGroup` was omitted, the `gid` would remain as 0 (root) and the process will
 be able to interact with files that are owned by the root(0) group and groups that have
-the required group permissions for the root (0) group.
+the required group permissions for the root (0) group. You can also see `groups`
+contains the group IDs which are specified by `fsGroup` and `supplementalGroups` other
+than `gid`.
 
 Exit your shell:
 
 ```shell
 exit
 ```
 
+### Implicit group memberships defined in `/etc/group` in the container image
+
+By default, kubernetes merges group information for the container's primary user defined in
+`/etc/group` in the container image.
+
+{{% code_sample file="pods/security/security-context-5.yaml" %}}
+
+In this configuration, it just specifies `runAsUser`, `runAsGroup` and `supplementalGroups`.
+However, you can see that the actual supplementary groups attached to the container process
+will include group IDs which come from `/etc/group` in the container image.
+
+Create the Pod:
+
+```shell
+kubectl apply -f https://k8s.io/examples/pods/security/security-context-5.yaml
+```
+
+Verify that the Pod's Container is running:
+
+```shell
+kubectl get pod security-context-demo
+```
+
+Get a shell to the running Container:
+
+```shell
+kubectl exec -it security-context-demo -- sh
+```
+
+Check the process identity:
+
+```shell
+$ id
+```
+
+The output is similar to this:
+
+```none
+uid=1000(user-defined-in-image) gid=3000 groups=3000,4000,50000(group-defined-in-image)
+```
+
+You can see `groups` includes group ID `50000`. This is because the user (`uid=1000(user-defined-in-image)`)
+belongs to the group `group-defined-in-image(gid=50000)` which is defined in `/etc/group` in the container image.
+
+Check the `/etc/group` in the container image:
+
+```shell
+$ cat /etc/group
+```
+
+You can see the group entry that `user-defined-in-image(uid=1000)` belongs to `group-defined-in-image(gid=50000)`.
+
+```none
+...
+group-defined-in-image:x:50000:user-defined-in-image
+```
+
+Exit your shell:
+
+```shell
+exit
+```
+
+{{<note>}}
+Implicitly _merged_ supplementary groups may cause security concerns particularly in accessing
+the volumes (see [kubernetes/kubernetes#112879](https://issue.k8s.io/112879) for details).
+If you want to avoid this. Please see the below section.
+{{</note>}}
+
+## Configure fine-grained SupplementalGroups control for a Pod {#supplementalgroupspolicy}
+
+{{< feature-state feature_gate_name="SupplementalGroupsPolicy" >}}
+
+This feature can be enabled by setting the `SupplementalGroupsPolicy`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for kubelet and
+kube-apiserver, and setting the `.spec.securityContext.supplementalGroupsPolicy` field for a pod.
+
+**supplementalGroupsPolicy** - `supplementalGroupsPolicy` defines behavior for calculating
+supplementary groups for the container processes in a pod.
+
+* _Merge_: The group membership defined in `/etc/group` for the container's primary user will be merged. If not specified, this policy will be applied.
+
+* _Strict_: it only attaches group IDs in `fsGroup`, `supplementalGroups`, or `runAsGroup` fields as the supplementary groups of the container processess. This means no group membership defined in `/etc/group` for the container's primary user will be merged.
+
+When the feature is enabled, it also exposes the process identity attached to the first container process of the container
+in `.status.containerStatuses[].user.linux` field. It would be helpful to detect if implicit group ID's are attached.
+
+{{% code_sample file="pods/security/security-context-6.yaml" %}}
+
+This pod manifest defines `supplementalGroupsPolicy=Strict`. You can see no group membership defined in `/etc/group` will be merged to the supplementary groups for container processes.
+
+Create the Pod:
+
+```shell
+kubectl apply -f https://k8s.io/examples/pods/security/security-context-6.yaml
+```
+
+Verify that the Pod's Container is running:
+
+```shell
+kubectl get pod security-context-demo
+```
+
+Check the process identity:
+
+```shell
+kubectl exec -it security-context-demo -- id
+```
+
+The output is similar to this:
+
+```none
+uid=1000(user-defined-in-image) gid=3000 groups=3000,4000
+```
+
+See the Pod's status:
+
+```shell
+kubectl get pod security-context-demo -o yaml
+```
+
+You can see `status.containerStatuses[].user.linux` field exposes the process identitiy
+attached to the first container process.
+
+```none
+...
+status:
+  containerStatuses:
+  - name: sec-ctx-demo
+    user:
+      linux:
+        gid: 3000
+        supplementalGroups:
+        - 3000
+        - 4000
+        uid: 1000
+...
+```
+
+{{<note>}}
+Please note that the values in `status.containerStatuses[].user.linux` field is _the firstly attached_
+process identity to the first container process in the container. If the container has sufficient privilege
+to call system calls related to process identity (e.g. [`setuid(2)`](https://man7.org/linux/man-pages/man2/setuid.2.html), [`setgid(2)`](https://man7.org/linux/man-pages/man2/setgid.2.html) or [`setgroups(2)`](https://man7.org/linux/man-pages/man2/setgroups.2.html), etc.),
+the container process can change its identity. Thus, the _actual_ process identity will be dynamic.
+{{</note>}}
+
+### Implementations {#implementations-supplementalgroupspolicy}
+
+{{% thirdparty-content %}}
+
+The following container runtimes are known to support fine-grained SupplementalGroups control.
+
+CRI-level:
+- [containerd](https://containerd.io/), since v2.0
+- [CRI-O](https://cri-o.io/), since v1.31
+
+You can see if the feature is supported in the node status.
+
+```yaml
+apiVersion: v1
+kind: Node
+...
+status:
+  features:
+    supplementalGroupsPolicy: true
+```
+
 ## Configure volume permission and ownership change policy for Pods
 
 {{< feature-state for_k8s_version="v1.23" state="stable" >}}

content/en/examples/pods/security/security-context-5.yaml

@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: security-context-demo
+spec:
+  securityContext:
+    runAsUser: 1000
+    runAsGroup: 3000
+    supplementalGroups: [4000]
+  containers:
+  - name: sec-ctx-demo
+    image: registry.k8s.io/e2e-test-images/agnhost:2.45
+    command: [ "sh", "-c", "sleep 1h" ]
+    securityContext:
+      allowPrivilegeEscalation: false

content/en/examples/pods/security/security-context-6.yaml

@@ -0,0 +1,16 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: security-context-demo
+spec:
+  securityContext:
+    runAsUser: 1000
+    runAsGroup: 3000
+    supplementalGroups: [4000]
+    supplementalGroupsPolicy: Strict
+  containers:
+  - name: sec-ctx-demo
+    image: registry.k8s.io/e2e-test-images/agnhost:2.45
+    command: [ "sh", "-c", "sleep 1h" ]
+    securityContext:
+      allowPrivilegeEscalation: false

content/en/examples/pods/security/security-context.yaml

@@ -7,6 +7,7 @@ spec:
     runAsUser: 1000
     runAsGroup: 3000
     fsGroup: 2000
+    supplementalGroups: [4000]
   volumes:
   - name: sec-ctx-vol
     emptyDir: {}