Does capabilities.drop: ALL make sense with privileged: true?

[Ais8Ooz8]

https://github.com/kubernetes-sigs/azuredisk-csi-driver/blob/master/charts/latest/azuredisk-csi-driver/templates/csi-azuredisk-node.yaml

[andyzhangx]

Good question! In Kubernetes, setting privileged: true in the securityContext gives the container almost all the capabilities of the host, effectively bypassing most security restrictions.

On the other hand, capabilities.drop: ALL attempts to drop all Linux capabilities from the container.

However, when privileged: true is set, the container runs with elevated privileges, and dropping capabilities generally has no effect because privileged mode overrides capability restrictions.

So, using capabilities.drop: ALL together with privileged: true does not make much sense, as the privileged flag grants all capabilities regardless of what you try to drop.

If your goal is to restrict capabilities, you should avoid using privileged: true and instead selectively add or drop capabilities.

[Ais8Ooz8]

@andyzhangx Is it true that we have no choice but to enable privileged: true in the CSI driver? Since mountPropagation: Bidirectional is used, it only works with privileged: true, right?

Bidirectional mount propagation can be dangerous. It can damage the host operating system and therefore it is allowed only in privileged containers. Familiarity with Linux kernel behavior is strongly recommended.

I'm confused by the use of such a wildcard instead of specific capabilities. Maybe some kind of enhancement is being developed in Kubernetes? Do you know anything about it?

[andyzhangx]

I think yes, that's the reason

[Ais8Ooz8]

@jsafrane @msau42 @saad-ali @xing-yang Hello, I need your help as sig-storage leads. As far as I understand, any CSI driver should work with privileged: true enabled, since it uses mountPropagation: Bidirectional for mountPath: /var/lib/kubelet. But in this case, apparmor is disabled, seccomp is disabled, all capabilities are enabled, and in short, the CSI driver gets the maximum possible privileges. Do you have any ideas on how to replace privileged: true with something more restrictive?

What if mountPropagation: Bidirectional checked for CAP_SYS_ADMIN instead of privileged: true? Although the Linux developers explicitly state "this capability is overloaded", "if you can possibly avoid it", I don't see them implementing the necessary privileges as a separate capability. Apparently, it's not that simple. But even switching to CAP_SYS_ADMIN instead of privileged: true will likely give us more security options. At a minimum, we'll be able to use seccomp and apparmor. Thank you in advance for any insights or suggestions you can provide.

[Ais8Ooz8]

By the way, I noticed that CSI drivers usually mount /var/lib/kubelet completely. Maybe it would be better to just mount /var/lib/kubelet/pods and /var/lib/kubelet/plugins, so access to /var/lib/kubelet/seccomp and /var/lib/kubelet/pki is restricted?