Container (Ansible Execution Environment) not allowed to connectto volume mounted ssh-agent socket, what is the correct/proper way?

[reenberg]

I am trying to run an Ansible EE container, which needs SSH credentials for the remote server. I would rather not mount in my ~/.ssh directory, but rather just start an ssh-agent for the duration, add they needed key, and expose socket while running the Ansible EE, and then clean it all up nicely.

Besides the fact that ansible-runner currently broken, and thus doesn't handle SSH keys for EE, it has historically made it work with --ipc=host, and most other solutions out there is to basically disable SELinux (as always...)

I tried the following, running Podman rootless from my regular user:

$ podman run --rm -it \
  -v ${SSH_AUTH_SOCK}:${SSH_AUTH_SOCK}:Z -e SSH_AUTH_SOCK=${SSH_AUTH_SOCK} \
  --user root \
  ubi9/python-312 \
  sh -c 'ls -alZ ${SSH_AUTH_SOCK}'
srw-------. 1 root root system_u:object_r:container_file_t:s0:c316,c940 0 Sep 16 20:48 /tmp/ssh-Xm2ZWUL1Bpoy/agent.1505382

$ podman run --rm -it \
  -v ${SSH_AUTH_SOCK}:${SSH_AUTH_SOCK}:Z -e SSH_AUTH_SOCK=${SSH_AUTH_SOCK} \
  --user root \
  ubi9/python-312 \
  ssh-add -L
Error connecting to agent: Permission denied

looking in audit.log, i see the AVC denial, as expected:

type=AVC msg=audit(1726523691.473:17372): avc:  denied  { connectto } for  pid=1781670 comm="ssh-add" path="/tmp/ssh-Xm2ZWUL1Bpoy/agent.1505382" scontext=system_u:system_r:container_t:s0:c446,c849 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=unix_stream_socket permissive=0

What is the correct and secure way to achieve this using Podman with SELinux enabled? Should this be filed as a issue against container-selinux, so that a rule could be added, to allow this, or is this an insanely stupid idea in the first place, and should the specific key be mounted RO into the container's ~/.ssh directory instead?

[GongT]

In my opinion, ssh-agent should never be used, anywhere. (reason: it's too easy to expose key to unwanted process/location, and it's hard to realize)
mount specific key file will much safer than use an agent.

But mount key file should hit another AVC denial, you still required to add some policy. LOL

[reenberg]

It is easy to shoot yourself in the foot no matter what gun you have in your hand, if you do stupid stuff.

I agree that if you blindly mount in the users "default" ssh-agent, then it could very well contain keys unrelated to the execution of the container, just af if you blindly mount in the user's entire ~/.ssh folder. So yeah, basically, everything that ansible-runner is doing today...

However I would argue that it is not necessarily worse nor better to use ssh-agent if a new one is created specifically for the execution of the container, with the needed keys added, or if only the specific keys and/or ssh config is mounted in.

However it doesn't make the use case any less relevant.
And I would highly assume that it is better to allow this kind of volume mount, than running the container without disabling selinux, running it as priviliged or with host IPC.

[GongT]

There are two things:

1. SELinux: you need to tell it "this program (in container) should allow to access ssh-keys". SELinux is too complicated, no way to have a general policy for this, so you must do it yourself.
2. Application: container app need a way to use ssh-keys from host. agent and mount files are both ok.

Copy/mount file is the better when no SELinux. That's definitely true.

With SELinux enabled, I think there is no different.

[reenberg]

Please not that i value your input.

I am not quite sure what you mean in 1, about needing to tell SELinux to allow access to the ssh-keys.
Volume mounting a single ssh-key into the containers as '~/.ssh/id_rsa', or the entire ~/.ssh folder works perfectly fine, without any SELinux configuration:

$ podman run --rm -it -v ~/.ssh:/root/.ssh:ro,Z -u root ubi9/python-312:latest ssh root@my-target-server
Enter passphrase for key '/root/.ssh/id_rsa': 
Last login: Thu Sep 12 23:25:52 2024 from A.B.C.D
[detached (from session main)]
Connection to my-target-server closed.

The primary issue would be the POSIX ownership of the files that is being mounted in (assuming they are mounted from the users home dir, and thus normally owned by her). But as ansible-runner more or less only works when running as root inside the container (sadly), then the POSIX permissions is no issue, as the host UID would normally map to UID 0 inside the container.
However if it was not running as root inside, but the default user created in the container, then the ownership could be "easily" fixed with --userns=keep-id:uid=1001,gid=0 (The UBI9 Python image has a default user of 1001:0 created). However using gid=0 requires a fix which is only available in podman 5.x to work.

I don't have podman 5.x on this machine, and the GID is less important in this example, so I am just using GID=1001 here:

$ podman run --rm -it -v ~/.ssh:/opt/app-root/src/.ssh:ro,Z --userns=keep-id:uid=1001,gid=1001 ubi9/python-312:latest ssh root@my-target-machine
...

Also i am unsure why you argue that there is no way to have a general SELinux policy for this?
I don't claim to be an SELinux expert, and the following TE rule could perhaps be made more specific(?) but it works:

allow container_t unconfined_t:unix_stream_socket connectto;

Loading this single allow rule, allows SSH inside the container to read the key from ssh-agent:

$ podman run --rm -ti -v ~/.ssh/known_hosts:/root/.ssh/known_hosts:ro,Z -v ${SSH_AUTH_SOCK}:${SSH_AUTH_SOCK}:Z -e SSH_AUTH_SOCK=${SSH_AUTH_SOCK} -u root ubi9/python-312:latest ssh root@my-target-server

Alternatively invoking Podman through ssh-agent is potentially a simple way to ensure no unintended keys are leaked, and that only the single specified key is exposed.

$ ssh-agent -t 10s bash -c 'ssh-add ~/.ssh/id_rsa && podman run ...'

Obviously one could use Udica to generate the rules, however that would require either ansible-runner to ship a custom SELinux modules and then label EE containers when running them, which I guess would be somewhat complicated to achieve as ansible-runner is typically recommended to be installed directly from PyPi or git, and only seems available as an RPM from the Fedora repositories.

So from my point of view, it basically boils down to whether the above TE allow rule could be made more specific, or if it would be useful as is (if it is not opening an unintended "can of worms").
At face value, it certainly seems better than the three alternatives (disable SELinux, priviledged, or host IPC).