-
Notifications
You must be signed in to change notification settings - Fork 648
Description
/kind bug
What happened:
We currently use manila-csi-plugin with the CEPHFS share protocol. We would like to upgrade both manila and cephfs-csi to keep in line with the deprecation of the old mounting syntax in cephfs (cf. ceph/ceph-csi#5090).
However after upgrading both csi-drivers I am confused on the correct way to mount shares using an alternate file system (i.e. one other than the default cephfs
filesystem). Previously we were passing the mds_namespace=my_fs
to the cephfs-{fuse/kernel}MountOptions
(as referenced in the manila documentation).
My understanding is that now we should use client_fs
in the case of mounting via fuse, however when passing this via mountOptions: [client_fs=my_fs]
or parameters.cephfs-fuseMountOptions: client_fs=my_fs
in my storage class it gets merged with the rest of options.
This means instead of getting something like --client-fs=my-fs -o noatime
in my mount command it looks like this -o noatime,client_fs=my-fs
and mounting fails with the following error:
ceph-fuse[223933]: starting ceph client
ceph-fuse[223933]: ceph mount failed with (2) No such file or directory
Warning FailedMount 3s kubelet MountVolume.MountDevice failed for volume "pvc-eac0a4e0-799f-4315-8379-f6ce2ff9e4ac" : rpc error: code = Internal desc = an error (exit status 1) occurred while running ceph-fuse args: [/var/lib/kubelet/plugins/kubernetes.io/csi/cephfs.manila.csi.openstack.org/6ed45902d15882166a8540eab426ea53eebe07e6adecc4f42585a039052fcc15/globalmount -m ***redacted*** -c /etc/ceph/ceph.conf -n client.pvc-eac0a4e0-799f-4315-8379-f6ce2ff9e4ac --keyfile=***stripped*** -r /volumes/_nogroup/fe01001f-1643-496a-9e43-0629385539a2/8665b574-a1d8-474f-93f1-7afc6fb4fe5b -o noatime,--client_fs=my_fs] stderr: 2025-09-15T13:30:38.969+0000 7fc831307580 -1 init, newargv = 0x55d3597039f0 newargc=15
2025-09-15T13:30:38.969+0000 7fc831307580 -1 init, args.argv = 0x55d3598543a0 args.argc=4
ceph-fuse[223966]: starting ceph client
ceph-fuse[223966]: ceph mount failed with (2) No such file or directory
What you expected to happen:
The alternate filesystem can be defined either via parameters.fsName
or via the mountOptions
in the storage class used to dynamically provision the share.
How to reproduce it:
-
Install the openstack-manila-csi:2.33.0 and ceph-csi-cephfs:3.15.0 helm charts with their respective attached configurations (ceph-csi-cephfs-config.yaml and openstack-manila-csi-config.yaml).
-
Attempt to mount a dynamically provisioned share using an alternate filesystem.
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: cephfs-alt-fs
parameters:
csi.storage.k8s.io/controller-expand-secret-name: os-trustee
csi.storage.k8s.io/controller-expand-secret-namespace: kube-system
csi.storage.k8s.io/node-publish-secret-name: os-trustee
csi.storage.k8s.io/node-publish-secret-namespace: kube-system
csi.storage.k8s.io/node-stage-secret-name: os-trustee
csi.storage.k8s.io/node-stage-secret-namespace: kube-system
csi.storage.k8s.io/provisioner-secret-name: os-trustee
csi.storage.k8s.io/provisioner-secret-namespace: kube-system
type: CephFS SSD MyFs
cephfs-mounter: fuse
mountOptions:
- "client_fs=my_fs"
provisioner: cephfs.manila.csi.openstack.org
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: cephfs-alt-fs
spec:
storageClassName: cephfs-alt-fs
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
name: nginx-cephfs-alt-fs
spec:
containers:
- name: nginx
image: nginx
volumeMounts:
- mountPath: /var/lib/www/html
name: my-vol
volumes:
- name: my-vol
persistentVolumeClaim:
claimName: cephfs-alt-fs
Anything else we need to know?:
Environment:
- openstack-cloud-controller-manager(or other related binary) version: v1.33.1
- OpenStack version: 6.0.1
- openstack-manila-csi:2.33.0
- ceph-csi-cephfs:3.15.0
Please could you clarify whether passing client_fs
to mount an alternate filesystem in cephfs via manila is supported?
If not I am more than happy to help contribute.
Thanks for your help in advanced
Cheers,
Jack