Question about running rootless without privileged pod in EKS

[lefterisALEX]

Hello, i am trying to run podman in a kubernetes cluster and facing some challenges running it as non root user and not privileged.
I followed the steps in this guide and these are the results i have:
Rootful/Rootless Podman with the privileged flag set
This looks is working for both root and podman user.

$ kubectl exec -it podman-priv -- sh                                                                                                                                                         
sh-5.2# podman run ubi8 echo hello
Resolved "ubi8" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull registry.access.redhat.com/ubi8:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob 2efec45cd878 done   | 
Copying config c70d72aaeb done   | 
Writing manifest to image destination
Storing signatures
WARN[0013] Path "/run/secrets/etc-pki-entitlement" from "/etc/containers/mounts.conf" doesn't exist, skipping 
WARN[0013] Path "/run/secrets/rhsm" from "/etc/containers/mounts.conf" doesn't exist, skipping 
hello
sh-5.2# 

Rootless Podman without the privileged flag
I would like to have this way working ,but looks like a syscall is not allowed?

$ kubectl exec -it no-priv -- sh                                                                                                                                        
sh-5.2$ id
uid=1000(podman) gid=1000(podman) groups=1000(podman)
sh-5.2$ podman run ubi8 echo hello
cannot clone: Operation not permitted
Error: cannot re-exec process

Rootful Podman without the privileged flag

$ kubectl exec -it no-priv-rootful -- sh                                                                                                                                                        

sh-5.2# podman run ubi8 echo hello
Resolved "ubi8" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull registry.access.redhat.com/ubi8:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob 2efec45cd878 done   | 
Copying config c70d72aaeb done   | 
Writing manifest to image destination
Storing signatures
WARN[0009] Path "/run/secrets/etc-pki-entitlement" from "/etc/containers/mounts.conf" doesn't exist, skipping 
WARN[0009] Path "/run/secrets/rhsm" from "/etc/containers/mounts.conf" doesn't exist, skipping 
Error: crun: create keyring `6cdc3d2f6d883a16e8b7f55a9c0f100991680edebc89cbbbfeed7253ee7bfc19`: Operation not permitted: OCI permission denied

Some more info:

1. K8S version: 1.25 ( EKS with self-managed nodes)
2. All pods are using the same seccompProfile.

  securityContext:
    seccompProfile:
      type: RuntimeDefault

[rhatdan]

cannot clone: Operation not permitted

Is caused by seccomp, I would figure that you are using seccom.json from Docker/Containerd, which is blocking the CLONE syscall. Podman/CRI-O allow the syscall by default.

[lefterisALEX]

Thanks for the reply @rhatdan , guess this seccomp profile is required then.