"fusermount3: mount failed: Operation not permitted" in unprivileged pod

[MetalPinguinInc]

Hi squat,

Thanks for making this plugin. It seems to be exactly what I need: mounting a remote filesystem over sshfs without using privileged/SYS_ADMIN pods. Unfortunately I cannot get it working. I have installed the daemon set using the example yaml from the README, but I get the error in the title when I am actually trying to use sshfs.

When I grant a pod the SYS_ADMIN capability it works as expected, but I am trying to get it working without that capability. Unfortunately the examples on how to use this plugin for unprivileged FUSE mounts are a bit scarce.

Yaml for pod:

apiVersion: v1
kind: Pod
metadata:
  name: sshfs-test-pod
  namespace: sshfstest  
  labels:
    app: sshfs-test
spec:
  nodeName: jackalope
  containers:
    - name: sshfs-test
      image: mcr.microsoft.com/dotnet/sdk:7.0.203
      command:
        - sh
      args:
        - '-c'
        - while true; do sleep 2; done
      resources: 
        limits:
          squat.ai/fuse: 1
      # securityContext:
      #   capabilities:
      #     add: ["SYS_ADMIN"]
  restartPolicy: Never

In the pod I execute:

apt update
apt install sshfs -y
mkdir /mnt/sshfs
sshfs <sshtestuser>@<domain>:<path/to/shared/folders> /mnt/sshfs/

The exact same commands work when the pod has SYS_ADMIN capabilities so I am sure there is no issue on the remote server regarding rights/firewalling etc.

Please let me know if you need additional info.

[squat]

Hi @MetalPinguinInc, traditionally the mount(2) system call, which is used by fuse/fusermount, requires CAP_SYS_ADMIN. Note that this is different from setting privileged: true in Kubernetes: it is more limited, does not provide access to host devices, etc.

If CAP_SYS_ADMIN is given to the container, the capabilities can be further limited by using a SECCOMP profile [0].

There are also some more novel approaches that permit mounting fuse without CAP_SYS_ADMIN, e.g. by the root user within a user NS [1] [2], however these are not available within Kubernetes.

[0] kubernetes/kubernetes#7890 (comment)
[1] torvalds/linux@4ad769f
[2] https://zameermanji.com/blog/2022/8/5/using-fuse-without-root-on-linux/

[MetalPinguinInc]

Hi @squat,

Thanks a lot for the quick reply. I had run into de seccomp solution before, but thought this project would make that obsolete. So it looks like I will need to either grant the pod SYS_ADMIN capabilities or use a custom seccomp profile then. Both are not ideal, but I understand this is out of scope for this project.

Thank you so much for the links you provided, I will read definitely read them for more background info. I will leave it up to you if you want to close this issue or if you want to resolve it in the future if Kubernetes ever allows fusermount with default seccomp. I am quite sure this issue and your reply will at least save some future readers a lot of frustration.