How to use rootless podman with another non-root user? (--user)

[artemdxc]

I need to start the rootless container from another non-root user (preferrably by its id), i.e.:

user@vbox:~$ podman run --user 20000 ( . . . )

where user is non-root user, 20000 is the id of another non-root user.

/etc/subuid and /etc/subgid contain the appropriate offset, smth like:

user:10000:65536

inside each of them.

First, I try the mentioned command in Ubuntu:

user@vbox:~$ podman run -dt --user 20000 localhost/test-container sleep 123

and it works correctly, namely:

user@vbox:~$ ps -C sleep -F -L
UID ( . . . ) CMD
29999 ( . . . ) sleep 123

user@vbox:~$ podman top -l huser user
HUSER USER
29999 20000

pay attention that

1. container's process has uid 20000,
2. the sleep 123 process executes from the uid 29999 from the point of view of the host OS, which is exactly "userid + offset - 1".

It works correctly (starts the process inside the container from user specified by the --user) at the following devices:

1. Raspberry Pi 4 with Ubuntu 22.04,
2. virtual machine with Ubuntu 22.04.

But then I try to do the same at the ARM_64 based board with the Linux OS. And it doesn't work at all for non-root and even for root:

[root@host ~]# podman run -dt --user 20000 localhost/test-container sleep 123

[root@host ~]# ps -C sleep -F -L
UID ( . . . ) CMD
root ( . . . ) sleep123

[from host OS] podman attach 81e1b3a212
[inside the container] id
<it shows "root" and id "0" when I expect "some-non-root" and id "20000">

(it starts process inside the container from the current user and doesn't take into account --user option)

I tried to use the same Linux kernel versions (5.4), almost-the-same podman versions (3.2.1 and 3.2.4) and almost-the-same versions of crun (1.4.5 and 1.5.0) at the ARM board and at the VM, but it still doesn't work at the board, while it works at the VM with Ubuntu even if versions of Linux, podman and crun differs.

So, can you suggest me what I am doing wrong and why the container doesn't start from user specified by the --user option?

[mheon]

Need more details on the arm64 environment - a podman info could help. Also, the output of /proc/self/uid_map and /proc/self/gid_map from inside the container would be useful.

[artemdxc]

podman info

host:
  arch: arm64
  buildahVersion: 1.21.3
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: Unknown
    path: /usr/sbin/conmon
    version: 'conmon version 2.0.29, commit: 7e6de6678f6ed8a18661e1d5721b81ccee293b9b'
  cpus: 2
  distribution:
    distribution: ptxdist
    version: 2018.07.0
  eventLogger: file
  hostname: host
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 200
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 10003
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.4.70
  linkmode: dynamic
  memFree: 7335936
  memTotal: 217174016
  ociRuntime:
    name: crun
    package: Unknown
    path: /usr/sbin/crun
    version: |-
      crun version 1.4.5
      commit: c381048530aa750495cf502ddb7181f2ded5b400
      spec: 1.0.0
      +SELINUX +APPARMOR +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /tmp/podman-run-10003/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: Unknown
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.1
  swapFree: 0
  swapTotal: 0
  uptime: 18m 56.7s
registries: {}
store:
  configFile: /opt/appl/.config/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 0
    stopped: 2
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: Unknown
      Version: |-
        fusermount3 version: 3.4.1
        fuse-overlayfs: version 1.8.2
        FUSE library version 3.4.1
        using FUSE kernel interface version 7.27
  graphRoot: /opt/appl/data/containers/storage/applexec
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 1
  runRoot: /tmp/podman-run-10003/containers
  volumePath: /opt/appl/data/containers/storage/applexec/volumes
version:
  APIVersion: 3.2.3
  Built: 0
  BuiltTime: Thu Jan  1 00:00:00 1970
  GitCommit: ""
  GoVersion: go1.16.5
  OsArch: linux/arm64
  Version: 3.2.3

uid_map and gid_map:

[user@host ~]$ podman run -dt -u 20000 docker.io/arm64v8/busybox:1.35.0
38a4d13b210d783d37289b5a28f982d48705591739c111b3715e5939dbfdf24a
[user@host ~]$ podman attach 3
/ # id
uid=0(root) gid=0(root)
/ # cat /proc/self/uid_map
         0      10003          1
         1     100000      65536
/ # cat /proc/self/gid_map
         0        200          1
         1     100000      65536

"user" has uid 10003 (at the host) and belongs to the group with gid 200

[artemdxc]

Also I used strace, analyzed obtained logs and found out the following:

• podman in Ubuntu (VM and Raspberry) invokes the setresuid syscall at some step with the correct uid:

3641  setresuid(40001, 40001, 40001)    = 0

(where 40001 is the uid specified with the --user);

• but podman at the other ARM_64 board doesn't invoke it;

• also if podman run (...) starts from the non-root there are the following invokations:

3605  setresgid(0, 0, 0)                = 0
3605  setresuid(0, 0, 0)                = 0

(both at the board and at the Ubuntu)

Thus, what reason could cause such behavior? Or, in other words,

1. what should I check to find out why setresuid(40001, 40001, 40001) isn't invoked at the ARM_64 board?
2. what is the reason for podman to switch into the root setresuid(0, 0, 0) ?

[mheon]

Any chance you can try with runc instead of crun? I suspect this bit is happening in the OCI runtime.

[artemdxc]

Yes, it works with runc, but doesn't with crun

[rhatdan]

@giuseppe PTAL