unknown option `--setuser app'

[lu-you]

apiVersion: v1
data:
  config.json: |-
    [
      {
        "clusterID": "f7f296cc-5f39-442b-a621-473a090200f4",
        "monitors": [
          "x.x.x.x:6789"
        ],
        "cephFS": {
           "fuseMountOptions": "--setuser appops"
       }
     }
    ]
kind: ConfigMap

pod error

2024-03-06T10:47:52.707+0000 7f57b6d23580 -1 fuse_lowlevel_new failed
  Warning  FailedMount  1s  kubelet  MountVolume.MountDevice failed for volume "pvc-38e12f4b-2b9f-4fd7-ac3d-8f157a78515a" : rpc error: code = Internal desc = an error (exit status 33) occurred while running ceph-fuse args: [/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-38e12f4b-2b9f-4fd7-ac3d-8f157a78515a/globalmount -m 10.18.112.6:6789,10.18.112.7:6789,10.18.112.8:6789 -c /etc/ceph/ceph.conf -n client.admin --keyfile=***stripped*** -r /volumes/csi/csi-vol-a7284e73-d845-47cd-9d61-3aa54024cee4/1157dc81-cbc6-401b-9300-53b9cebdabe3 -o nonempty,--setuser appops,debug --client_mds_namespace=yidunfs] stderr: 2024-03-06T10:47:55.826+0000 7fcf87b33580 -1 init, newargv = 0x55e264102120 newargc=17
ceph-fuse[1139877]: starting ceph client
fuse: unknown option `--setuser appops'
ceph-fuse[1139877]: fuse failed to start
2024-03-06T10:47:55.831+0000 7fcf87b33580 -1 fuse_lowlevel_new faile

Backgroup:
my pod run user is app, but my cephfs-csi created pvc storage is root
so ,I hope the running identity I created is a normal user app, not root, how to solve it

[nixpanic]

Can you try with --setuser=appops and try again?

[lu-you]

--setuser=appops

yes

[lu-you]

Can you try with --setuser=appops and try again?

fuse: unknown option --setuser=appops' ceph-fuse[2429850]: fuse failed to start 2024-03-06T12:08:30.220+0000 7fc15c76b580 -1 fuse_lowlevel_new failed Warning FailedMount 3s kubelet MountVolume.MountDevice failed for volume "pvc-cda06ad2-f2d9-4140-a132-88e954a2b409" : rpc error: code = Internal desc = an error (exit status 33) occurred while running ceph-fuse args: [/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-cda06ad2-f2d9-4140-a132-88e954a2b409/globalmount -m 10.18.112.6:6789,10.18.112.7:6789,10.18.112.8:6789 -c /etc/ceph/ceph.conf -n client.admin --keyfile=***stripped*** -r /volumes/csi/csi-vol-37d8685f-b1a9-4e7f-9ca4-b54a3b832720/0b8aafef-dc9b-4a9b-ac82-13a6942cc616 -o nonempty,--setuser=appops,debug --client_mds_namespace=yidunfs] stderr: 2024-03-06T12:08:32.326+0000 7fcfebc9d580 -1 init, newargv = 0x55e2b1e69120 newargc=17 ceph-fuse[2430001]: starting ceph client fuse: unknown option --setuser=appops'
ceph-fuse[2430001]: fuse failed to start
2024-03-06T12:08:32.331+0000 7fcfebc9d580 -1 fuse_lowlevel_new failed
Warning FailedMount 0s kubelet MountVolume.MountDevice failed for volume "pvc-cda06ad2-f2d9-4140-a132-88e954a2b409" : rpc error: code = Internal desc = an error (exit status 33) occurred while running ceph-fuse args: [/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-cda06ad2-f2d9-4140-a132-88e954a2b409/globalmount -m 10.18.112.6:6789,10.18.112.7:6789,10.18.112.8:6789 -c /etc/ceph/ceph.conf -n client.admin --keyfile=stripped -r /volumes/csi/csi-vol-37d8685f-b1a9-4e7f-9ca4-b54a3b832720/0b8aafef-dc9b-4a9b-ac82-13a6942cc616 -o nonempty,--setuser=appops,debug --client_mds_namespace=yidunfs] stderr: 2024-03-06T12:08:35.450+0000 7f9c94bc3580 -1 init, newargv = 0x564cbf5ce120 newargc=17
ceph-fuse[2430155]: starting ceph client
fuse: unknown option `--setuser=appops'
ceph-fuse[2430155]: fuse failed to start
2024-03-06T12:08:35.455+0000 7f9c94bc3580 -1 fuse_lowlevel_new failed

And my host ceph-fuse version v14, cephcsi container ceph-fuse version is v18

[lu-you]

Can you try with --setuser=appops and try again?

And k8s version is 1.21，cephfs-csi is latest

[lu-you]

Can you try with --setuser=appops and try again?

is it because of the reason of ',' ?

-o nonempty --setuser appops --setgroup appops

[lu-you]

Nomal log:
I0307 05:57:27.781508 2375361 cephcmds.go:105] ID: 9 Req-ID: 0001-0024-f7f296cc-5f39-442b-a621-473a090200f4-0000000000000001-e0f9f6b2-fec4-41d0-a8df-8206ef21ea83 command succeeded: ceph-fuse [/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-4b7243cb-eb7d-4414-94db-ac32cd71efec/globalmount -m 10.18.112.6:6789,10.18.112.7:6789,10.18.112.8:6789 -c /etc/ceph/ceph.conf -n client.admin --keyfile=stripped -r /volumes/csi/csi-vol-e0f9f6b2-fec4-41d0-a8df-8206ef21ea83/2f404da2-8b99-493f-a6d7-c6fefc843564 -o nonempty,debug --client_mds_namespace=yidunfs]

Error log:
E0307 06:03:17.137538 1668133 utils.go:203] ID: 11 Req-ID: 0001-0024-f7f296cc-5f39-442b-a621-473a090200f4-0000000000000001-0392f0a1-319a-48ec-a147-52a780b05b06 GRPC error: rpc error: code = Internal desc = an error (exit status 33) occurred while running ceph-fuse args: [/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-23afd039-d812-458e-8881-418b7bcb9103/globalmount -m 10.18.112.6:6789,10.18.112.7:6789,10.18.112.8:6789 -c /etc/ceph/ceph.conf -n client.admin --keyfile=stripped -r /volumes/csi/csi-vol-0392f0a1-319a-48ec-a147-52a780b05b06/5f488d47-d616-4a19-af67-88052479fe45 -o nonempty,debug,--setuser 60001 --client_mds_namespace=yidunfs] stderr: 2024-03-07T06:03:16.127+0000 7f2d4b649580 -1 init, newargv = 0x5591bfad0120 newargc=17
ceph-fuse[1701898]: starting ceph client
fuse: unknown option `--setuser 60001'
ceph-fuse[1701898]: fuse failed to start
2024-03-07T06:03:16.132+0000 7f2d4b649580 -1 fuse_lowlevel_new failed

So , I strongly suspect that the problem is caused by ","

[nixpanic]

maybe try without spaces and leading --, like setuser=appops,setgroup=appops.

[lu-you]

maybe try without spaces and leading --, like setuser=appops,setgroup=appops.

No, I tried

you can have a look code，
https://github.com/ceph/ceph-csi/blob/devel/internal/util/util.go#L357 or

ceph-csi/internal/cephfs/mounter/fuse.go

Line 50 in 3410687

func mountFuse(ctx context.Context, mountPoint string, cr *util.Credentials, volOptions *store.VolumeOptions) error {

[lu-you]

maybe try without spaces and leading --, like setuser=appops,setgroup=appops.

And i think it's still a problem ---> ","

[nixpanic]

The -o option can take a single string with options separated by ,, so -o setuser=appops,setgroup=appops should be valid.

[lu-you]

The -o option can take a single string with options separated by ,, so -o setuser=appops,setgroup=appops should be valid.

Warning FailedMount 2s kubelet MountVolume.MountDevice failed for volume "pvc-f1f5354a-42be-434e-8f05-ea1014f6025b" : rpc error: code = Internal desc = an error (exit status 33) occurred while running ceph-fuse args: [/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-f1f5354a-42be-434e-8f05-ea1014f6025b/globalmount -m 10.18.112.6:6789,10.18.112.7:6789,10.18.112.8:6789 -c /etc/ceph/ceph.conf -n client.admin --keyfile=stripped -r /volumes/csi/csi-vol-12471e9c-9ca3-4ed3-a16b-88994c7cd756/4e7381ea-1b91-4147-9092-71b448d97213 -o nonempty,debug,setuser=appops --client_mds_namespace=yidunfs] stderr: 2024-03-07T14:24:26.938+0000 7f24ebff9580 -1 init, newargv = 0x557f5cf50120 newargc=17
ceph-fuse[649388]: starting ceph client
fuse: unknown option `setuser=appops'
ceph-fuse[649388]: fuse failed to start
2024-03-07T14:24:26.943+0000 7f24ebff9580 -1 fuse_lowlevel_new failed

Warning FailedMount 1s kubelet MountVolume.MountDevice failed for volume "pvc-bcc49106-8af5-4c16-9fc1-041a51553ada" : rpc error: code = Internal desc = an error (exit status 33) occurred while running ceph-fuse args: [/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-bcc49106-8af5-4c16-9fc1-041a51553ada/globalmount -m 10.18.112.6:6789,10.18.112.7:6789,10.18.112.8:6789 -c /etc/ceph/ceph.conf -n client.admin --keyfile=stripped -r /volumes/csi/csi-vol-63904da8-2355-4cee-b84f-fc1ddd463cac/2015d940-9266-4c61-b192-3c5a30322560 -o nonempty,debug,setuser appops --client_mds_namespace=yidunfs] stderr: 2024-03-07T14:25:49.971+0000 7feaf8762580 -1 init, newargv = 0x56433e219120 newargc=17
ceph-fuse[652386]: starting ceph client
fuse: unknown option `setuser appops'
ceph-fuse[652386]: fuse failed to start
2024-03-07T14:25:49.976+0000 7feaf8762580 -1 fuse_lowlevel_new failed

look at it is , I tried, really can't

[nixpanic]

that is unfortunate, it seems the -o option does not accept the setuser parameter 😞

I do not see a setuser option in the cephfs-fuse manual, are you certain that the option is supported?

[lu-you]

that is unfortunate, it seems the -o option does not accept the setuser parameter 😞

I do not see a setuser option in the cephfs-fuse manual, are you certain that the option is supported?

ceph-fuse -h
usage: ceph-fuse [-n client.username] [-m mon-ip-addr:mon-port] [OPTIONS]
--client_mountpoint/-r <sub_directory>
use sub_directory as the mounted root, rather than the full Ceph tree.

usage: ceph-fuse mountpoint [options]

general options:
-o opt,[opt...] mount options
-h --help print help
-V --version print version

FUSE options:
-d -o debug enable debug output (implies -f)
-f foreground operation
-s disable multi-threaded operation

--conf/-c FILE read configuration from the given configuration file
--id ID set ID portion of my name
--name/-n TYPE.ID set name
--cluster NAME set cluster name (default: ceph)
--setuser USER set uid to user or uid (and gid to user's gid)
--setgroup GROUP set gid to group or gid
--version show version and quit

Aha，I checked the ceph-use -h

Can this parameter be supported?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation.