The user can fail to write to a volume even if fsGroup is properly set in securityContext

[mpatlasov]

What happened:

Even if user explicitly requires the same fsGroup as runAsUser and runAsGroup for a Pod:

spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1002
    runAsGroup: 1002
    fsGroup: 1002

he/she will fail to write to the volume:

bash-5.2$ touch /mnt/claim/FILE
touch: cannot touch '/mnt/claim/FILE': Permission denied

if StorageClass does not set uid/gid or dir_mode or noperm:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: samba
provisioner: smb.csi.k8s.io
parameters:
  source: //samba-server.samba-server.svc.cluster.local/share
  csi.storage.k8s.io/provisioner-secret-name: smbcreds
  csi.storage.k8s.io/provisioner-secret-namespace: samba-server
  csi.storage.k8s.io/node-stage-secret-name: smbcreds
  csi.storage.k8s.io/node-stage-secret-namespace: samba-server
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
  - file_mode=0777
  - mfsymlinks
  - cache=strict
  - noserverino

What you expected to happen:

If user requests the same fsGroup as runAsGroup in securityContext section of a pod definition, SMB CSI driver must construct mount options letting full access for that volume.

How to reproduce it:

Create StorageClass:

kubectl apply -f - <<EOF
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: samba
provisioner: smb.csi.k8s.io
parameters:
  source: //samba-server.samba-server.svc.cluster.local/share
  csi.storage.k8s.io/provisioner-secret-name: smbcreds
  csi.storage.k8s.io/provisioner-secret-namespace: samba-server
  csi.storage.k8s.io/node-stage-secret-name: smbcreds
  csi.storage.k8s.io/node-stage-secret-namespace: samba-server
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
  - file_mode=0777
  - mfsymlinks
  - cache=strict
  - noserverino
EOF

for a samba server from https://github.com/samba-in-kubernetes/samba-container .

Create a Pod requesting samba PV:

kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Mi
  storageClassName: samba
---
apiVersion: v1
kind: Pod
metadata:
  name: fedora
  labels:
    app: fedora
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1002
    runAsGroup: 1002
    fsGroup: 1002
  volumes:
  - name: claim2-vol
    persistentVolumeClaim:
      claimName: claim
  containers:
  - image: registry.fedoraproject.org/fedora-minimal
    command:
      - "sleep"
      - "604800"
    imagePullPolicy: IfNotPresent
    name: fedora
    securityContext:
      allowPrivilegeEscalation: false
      seccompProfile:
        type: RuntimeDefault
      capabilities:
        drop:
        - "ALL"
    volumeMounts:
      - mountPath: "/mnt/claim"
        name: claim2-vol
  restartPolicy: Always

Verify that the volume is not write-able from the Pod:

$ kubectl exec -it fedora -- bash
bash-5.2$ touch /mnt/claim/FILE
touch: cannot touch '/mnt/claim/FILE': Permission denied

Anything else we need to know?:

Environment:

• CSI Driver version: Image manually built by docker build ... -f ./cmd/smbplugin/Dockerfile . from the top of upstream master branch (commit 4d10e97 of https://github.com/kubernetes-csi/csi-driver-smb.git)

• Kubernetes version (use kubectl version):

Client Version: v1.30.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.3

[andyzhangx]

@mpatlasov can you provide the csi driver logs on the node? kubelet should Delegate fsGroup to CSI Driver in NodeStageVolume, thus you don't need to set gid in storage class:

https://kubernetes-csi.github.io/docs/support-fsgroup.html#delegate-fsgroup-to-csi-driver

[mpatlasov]

@andyzhangx , csi driver uses Delegate fsGroup correctly:

I0910 10:27:28.655774       1 nodeserver.go:209] NodeStageVolume: targetPath(/var/lib/kubelet/plugins/kubernetes.io/csi/smb.csi.k8s.io/4a32186b174aa41c2369dd747134fc00cb1514550a5e6e37f860ba407eab6120/globalmount) volumeID(samba-server.samba-server.svc.cluster.local/share#pvc-62d25ec6-c20e-45f8-815e-749c7df44b9a##) context(map[csi.storage.k8s.io/pv/name:pvc-62d25ec6-c20e-45f8-815e-749c7df44b9a csi.storage.k8s.io/pvc/name:claim csi.storage.k8s.io/pvc/namespace:default source://samba-server.samba-server.svc.cluster.local/share storage.kubernetes.io/csiProvisionerIdentity:1725963559773-1299-smb.csi.k8s.io subdir:pvc-62d25ec6-c20e-45f8-815e-749c7df44b9a]) mountflags([file_mode=0777 mfsymlinks cache=strict noserverino]) mountOptions([file_mode=0777 mfsymlinks cache=strict noserverino gid=1002])

adding gid=1002 to mount options. And gid=1002 is visible inside the Pod:

$ oc exec -it fedora -- bash
bash-5.2$ grep claim /proc/mounts
//samba-server.samba-server.svc.cluster.local/share/pvc-62d25ec6-c20e-45f8-815e-749c7df44b9a /mnt/claim cifs rw,relatime,vers=3.1.1,cache=strict,username=sambauser,uid=0,noforceuid,gid=1002,noforcegid,addr=172.30.35.246,file_mode=0777,dir_mode=0755,soft,nounix,mapposix,mfsymlinks,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1 0 0

However, the mount point is not writeable for the group:

bash-5.2$ ls -ld /mnt/claim 
drwxr-xr-x. 2 root 1002 0 Sep 10 13:35 /mnt/claim

I cannot see any workaround or solution to overcome this issue. Do you have any?

[mpatlasov]

@andyzhangx I'd appreciate your feedback on #848 . It fixes the issue by setting file_mode and dir_mode mount options. If this approach is tolerable, I'll add unit-tests to the PR. Thanks!