"/sys" caused: mount through procfd: operation not permitted: OCI permission denied #19524
Replies: 9 comments 2 replies
-
|
Can you test the with the latest version. Did you try it with crun instead of runc? cc @giuseppe |
Beta Was this translation helpful? Give feedback.
-
Seems like [cloud-user@preserve-olm-env2 jian]$ podman run --rm -ti --entrypoint /bin/bash registry.ci.openshift.org/ocp/4.14@sha256:8235c041ce1cb343b27301743414e31b7ab0fa9c57a0217fba4cd892d32e3e42
Error: runc: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:75: mounting "sysfs" to rootfs at "/sys" caused: mount through procfd: operation not permitted: OCI permission denied
[cloud-user@preserve-olm-env2 jian]$
[cloud-user@preserve-olm-env2 jian]$ podman --runtime crun run --rm -ti --entrypoint /bin/bash registry.ci.openshift.org/ocp/4.14@sha256:8235c041ce1cb343b27301743414e31b7ab0fa9c57a0217fba4cd892d32e3e42
[root@8ca488ca7e98 /]#
Will try it later. |
Beta Was this translation helpful? Give feedback.
-
|
Upgrade [cloud-user@preserve-olm-env2 podman]$ podman version
Client: Podman Engine
Version: 4.5.1
API Version: 4.5.1
Go Version: go1.20.4
Built: Thu Jun 15 18:01:33 2023
OS/Arch: linux/amd64But, still encounter this issue with [cloud-user@preserve-olm-env2 podman]$ podman run --rm -ti --entrypoint /bin/bash registry.ci.openshift.org/ocp/4.14@sha256:8235c041ce1cb343b27301743414e31b7ab0fa9c57a0217fba4cd892d32e3e42
Error: runc: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:75: mounting "sysfs" to rootfs at "/sys" caused: mount through procfd: operation not permitted: OCI permission denied |
Beta Was this translation helpful? Give feedback.
-
|
@giuseppe PTAL |
Beta Was this translation helpful? Give feedback.
-
|
Likely there is not a fully visible /sys mount available in your environment. crun has a fallback to bind mount /sys if a sysfs mount could not be used while runc fails. What is the environment where you are running your container? Could you share the output of |
Beta Was this translation helpful? Give feedback.
-
|
Sure, as follows, [cloud-user@preserve-olm-env2 podman]$ cat /etc/redhat-release
Red Hat Enterprise Linux release 8.4 (Ootpa)
[cloud-user@preserve-olm-env2 podman]$ uname -a
Linux preserve-olm-env2 4.18.0-287.el8.dt4.x86_64 #1 SMP Thu Feb 18 13:31:55 EST 2021 x86_64 x86_64 x86_64 GNU/Linux
[cloud-user@preserve-olm-env2 podman]$ cat /proc/self/mountinfo
22 99 0:21 / /sys rw,nosuid,nodev,noexec,relatime shared:3 - sysfs sysfs rw,seclabel
23 99 0:5 / /proc rw,nosuid,nodev,noexec,relatime shared:27 - proc proc rw
24 99 0:6 / /dev rw,nosuid shared:23 - devtmpfs devtmpfs rw,seclabel,size=8071292k,nr_inodes=2017823,mode=755
25 22 0:7 / /sys/kernel/security rw,nosuid,nodev,noexec,relatime shared:4 - securityfs securityfs rw
26 24 0:22 / /dev/shm rw,nosuid,nodev shared:24 - tmpfs tmpfs rw,seclabel
27 24 0:23 / /dev/pts rw,nosuid,noexec,relatime shared:25 - devpts devpts rw,seclabel,gid=5,mode=620,ptmxmode=000
28 99 0:24 / /run rw,nosuid,nodev shared:26 - tmpfs tmpfs rw,seclabel,mode=755
29 22 0:25 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:5 - tmpfs tmpfs ro,seclabel,mode=755
30 29 0:26 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,seclabel,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd
31 22 0:27 / /sys/fs/pstore rw,nosuid,nodev,noexec,relatime shared:18 - pstore pstore rw,seclabel
32 22 0:28 / /sys/fs/bpf rw,nosuid,nodev,noexec,relatime shared:19 - bpf bpf rw,mode=700
33 29 0:29 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:7 - cgroup cgroup rw,seclabel,devices
34 29 0:30 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:8 - cgroup cgroup rw,seclabel,cpu,cpuacct
35 29 0:31 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime shared:9 - cgroup cgroup rw,seclabel,pids
36 29 0:32 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:10 - cgroup cgroup rw,seclabel,net_cls,net_prio
37 29 0:33 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:11 - cgroup cgroup rw,seclabel,freezer
38 29 0:34 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:12 - cgroup cgroup rw,seclabel,memory
39 29 0:35 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:13 - cgroup cgroup rw,seclabel,perf_event
40 29 0:36 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:14 - cgroup cgroup rw,seclabel,cpuset
41 29 0:37 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:15 - cgroup cgroup rw,seclabel,blkio
42 29 0:38 / /sys/fs/cgroup/rdma rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,seclabel,rdma
43 29 0:39 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime shared:17 - cgroup cgroup rw,seclabel,hugetlb
44 22 0:12 / /sys/kernel/tracing rw,relatime shared:20 - tracefs none rw,seclabel
96 22 0:41 / /sys/kernel/config rw,relatime shared:21 - configfs configfs rw
99 1 252:3 / / rw,relatime shared:1 - xfs /dev/vda3 rw,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota
102 99 0:40 / /var/lib/nfs/rpc_pipefs rw,relatime shared:2 - rpc_pipefs rpc_pipefs rw
45 22 0:20 / /sys/fs/selinux rw,relatime shared:22 - selinuxfs selinuxfs rw
46 23 0:42 / /proc/sys/fs/binfmt_misc rw,relatime shared:28 - autofs systemd-1 rw,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19158
47 24 0:43 / /dev/hugepages rw,relatime shared:29 - hugetlbfs hugetlbfs rw,seclabel,pagesize=2M
48 24 0:19 / /dev/mqueue rw,relatime shared:30 - mqueue mqueue rw,seclabel
49 22 0:8 / /sys/kernel/debug rw,relatime shared:31 - debugfs debugfs rw,seclabel
50 22 0:44 / /sys/fs/fuse/connections rw,relatime shared:32 - fusectl fusectl rw
119 99 252:17 / /data rw,relatime shared:65 - xfs /dev/vdb1 rw,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota
122 99 252:2 / /boot/efi rw,relatime shared:67 - vfat /dev/vda2 rw,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro
396 46 0:48 / /proc/sys/fs/binfmt_misc rw,relatime shared:193 - binfmt_misc binfmt_misc rw
418 28 0:24 /netns /run/netns rw,nosuid,nodev shared:26 - tmpfs tmpfs rw,seclabel,mode=755
350 28 0:47 / /run/user/1000 rw,nosuid,nodev,relatime shared:187 - tmpfs tmpfs rw,seclabel,size=1621128k,mode=700,uid=1000,gid=1000 |
Beta Was this translation helpful? Give feedback.
-
|
Hi @giuseppe any updates? Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
/sys/fs/cgroup has a read only bind mount so the kernel blocks mounting a fresh sys. The error from the kernel is expected. crun has a fallback in this case, that is the difference with runc. I don't think podman should try to circumvent it, because it requires parsing the mounts table and that is known to be a relatively slow operation. My suggestion is either use crun or add an explicit bind mount for /sys, |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for your clarification! I'm clear now. [cloud-user@preserve-olm-env2 ~]$ mount | grep '/sys/fs/cgroup'
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,devices)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpu,cpuacct)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,net_cls,net_prio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,freezer)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,memory)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,perf_event)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpuset)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,blkio)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,rdma)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,hugetlb)
sudo groupadd cgroup_users
sudo usermod -a -G cgroup_users <xxx>
sudo chown -R root:cgroup_users /sys/fs/cgroup
sudo chmod -R g+rw /sys/fs/cgroupBut, anyway, it works well with [cloud-user@preserve-olm-env2 ~]$ podman run --rm -v /sys:/sys -ti --entrypoint /bin/bash registry.ci.openshift.org/ocp/4.14@sha256:8235c041ce1cb343b27301743414e31b7ab0fa9c57a0217fba4cd892d32e3e42
[root@a6e13cce33ae /]# |
Beta Was this translation helpful? Give feedback.
-
|
Moving this to a discussion. You could also open an issue with runc, to see if those developers are interested in the change that crun has done. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the suggestion! Here: opencontainers/runc#3970 |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Issue Description
Podman runfailed at the followings,Steps to reproduce the issue
Steps to reproduce the issue
podman run --group-add keep-groups --privileged --rm -ti quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:xxxDescribe the results you received
It works well with
sudo.Describe the results you expected
Run it well without
sudo.podman info output
Podman in a container
No
Privileged Or Rootless
None
Upstream Latest Release
No
Additional environment details
Additional environment details
Additional information
Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting
Beta Was this translation helpful? Give feedback.
All reactions