Is it possible to start rootfull podman backend but connect with unprivileged user to run podman-remote?

[qixiang]

Hi podman experts,

There is a remote-client tutorial doc, it covers running rootless podman backend and connect it to run podman-remote, my requirement is a little different:

1. I want to run rootfull podman backend
2. I want to connect the rootfull backend with an unprivileged user

Is it possible to start rootfull podman backend and grant full permissions to unprivileged users, or just start rootfull podman backend with unprivileged users?

I'm not sure whether this makes sense and want to ask for your opinions, if that is possible, could you let me how can I achieve this?

Steps to reproduce the issue:

1. Start rootfull podman backend on server side

[root@server ~]# systemctl start podman.socket
[root@server ~]# systemctl status podman.socket
● podman.socket - Podman API Socket
   Loaded: loaded (/usr/lib/systemd/system/podman.socket; disabled; vendor preset: disabled)
   Active: active (listening) since Tue 2022-01-18 06:49:59 EST; 3s ago
     Docs: man:podman-system-service(1)
   Listen: /run/podman/podman.sock (Stream)
   CGroup: /system.slice/podman.socket

2. From client side, add podman-remote session with an unprivileged user ssh connection (vagrant in this case).

[client]$ podman-remote system connection add rootfull --identity /local/vagrant/private_key ssh://[email protected]/run/podman/podman.sock
[client]$ podman-remote system connection list
Name        Identity                    URI
rootfull*   /local/vagrant/private_key  ssh://[email protected]:22/run/podman/podman.sock

3. From client side, run podman-remote command

[client]$ podman-remote -c rootfull ps
Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM
Error: unable to connect to Podman socket: Get "http://d/v3.4.4/libpod/_ping": ssh: rejected: connect failed (open failed)

Describe the results you received:
Can't run podman-remote commands with unprivileged user when podman backend is running in rootfull mode.

Describe the results you expected:
podman-remote commands can be executed with unprivileged user podman backend is running in rootfull mode.

Additional information you deem important (e.g. issue happens only occasionally):

I believe podman version doesn't matter here, so not including the info.

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

[rhatdan]

You could chown the permission on the rootfull podman.sock to be accessible by group Say the podman group, then you could add all users to the podman group, and setup podman-remote to use that socket. This would mimic what Docker does, but I am not crazy about it. If you make the change the following articles would apply to Podman as well as Docker.

https://projectatomic.io/blog/2015/08/why-we-dont-let-non-root-users-run-docker-in-centos-fedora-or-rhel/
https://danwalsh.livejournal.com/78373.html

[qixiang]

By default, you would need to set up your SSH connection to the root user to be able to use the rootful service. There might be a trick (like the chown that Dan mentioned) but I am personally not familiar with this.

If we're using ssh connection with root, that is actually the privileged user.
root + rootfull backend works, unprivileged user + rootless backend (of course with the same user) also works, but I want to use unprivileged user + rootfull backend.

[ashley-cui]

Edited since I read it wrong: Oh, yeah, I'm just saying that is the default, and I'm not sure about the configs that would make your situation work. Sorry for the mixup!

[qixiang]

No worries, the requirement is a little unusual, so it's easy to make some confusions and thanks for looking on this.

I believe if we can talk to the rootfull podman socket with unprivileged user on the same host (without sudo), the podman-remote should also work, so I'd just simplify my question to "how to grant unprivileged user with the permissions to rootfull podman socket".

1. Start rootfull podman backend, and we also grant rw permssions to vagrant group.

[root@server ~]# chown root.vagrant /run/podman/podman.sock
[root@server ~]# ls -l /run/podman/podman.sock
srw-rw----. 1 root vagrant 0 Jan 18 07:53 /run/podman/podman.sock

2. On the same host, talk to podman backend via the socket with an unprivileged user (which group is vagrant)

[vagrant@server ~]$ curl --unix-socket /run/podman/podman.sock http://localhost/_ping
curl: (7) Couldn't connect to server

Note: of course we can issue the curl command with sudo and it works, however that's not possible while using podman-remote, so that's not a proper solution.

$ sudo curl --unix-socket /run/podman/podman.sock http://localhost/_ping
OK

[afbjorklund]

Looks like you forgot to chown the /run/podman directory ? But should be done with systemd, to remain after restart.

[qixiang]

@afbjorklund you're right, the directory permission was not changed, I only changed permission of the socket, (as you suggested this should be managed by systemd, but I'm just changing it by manual temporarily), now it works for both unprivileged user on same host and podman-remote from remote hosts, thank you.

[root@server ~]# chown -R root.vagrant /run/podman
[root@server ~]# chmod g+rwx /run/podman
[root@server ~]# ls -ld /run/podman
drwxrwx---. 2 root vagrant 60 Jan 18 10:10 /run/podman

# on same host, with unprivileged user
[vagrant@server ~]$ curl --unix-socket /run/podman/podman.sock http://localhost/_ping
OK

# from remote host, with unprivileged user ssh connection
[client]$ podman-remote system connection list
Name        Identity                    URI
rootfull*   /local/vagrant/private_key  ssh://[email protected]:22/run/podman/podman.sock
[client]$ podman-remote ps
CONTAINER ID  IMAGE       COMMAND     CREATED     STATUS      PORTS       NAMES

[afbjorklund]

We used to do this with sudo, but when letting podman handle the ssh connection it is possible to use a root-equivalent group instead.

• Alternative to sudo, for remote podman #6809

Basically you need to configure systemd to create both the directory and the socket, using the "podman" group (instead of "root" group)

When using "podman machine", it should set up a "system connection" of each kind - so that you only need to switch --connection

[qixiang]

thank you, this sounds interesting, I don't fully understand it at this moment, but I'll read the mentioned links carefully and give it a try tomorrow.

[afbjorklund]

This is the implementation used in minikube, which uses podman when running Kubernetes with the CRI-O container runtime.

It doesn't use Vagrant, but there is a similar setup. The user environment is set up using the minikube podman-env command.

[qixiang]

I marked @afbjorklund 's reply as answered, but please also check the other answer which is just changing the permission of /run/podman and the socket in it, and keep in mind there may be some security issues with that approach (grant full permissions to unprivileged users).

[afbjorklund]

You could also change the VM and SSH configuration, and allow root to login directly.

This involves changing /etc/ssh/sshd_config and /root/.ssh/authorized_keys, if desired.

[afbjorklund]

there may be some security issues with that approach

you are basically recreating the Docker daemon, so yeah