Docker/Podman: full feature parity

[spookyvision]

Is your feature request related to a problem? Please describe

Podman is intended to be a drop-in replacement for Docker, but I can't use it in pyinfra docker operations/facts because the docker executable name is hardcoded as docker. My current workaround is a symbolic link /usr/local/bin/docker ➡️ /usr/bin/podman, but that feels hacky and has to be done manually for each host in the repository.

A symlink like that also complicates matters if one intends to run docker and podman on the same machine (not sure I'd recommend that anyway, but...)

Describe the solution you'd like

(edit: based on this issue's discussion, I changed my position on how to achieve feature parity. What follows below is the revised wording, the original preserved at the bottom.)
ideal solution:

• podman facts, identical in features to docker facts
• podman operations, identical in features to docker operations

In terms of implementation this can likely be achieved with something like an abstract DockerCompatibleContainer base classes or mixins, since Podman's CLI API is supposed to be identical do Docker's (though subtle differences do exist)

(original/outdated solution below)
Add a docker_executable inventory key that's passed to docker facts (which in turn enables the use of docker operations, since they rely on docker facts). Ideally the dockerssh connector would also make use of it.

[wowi42]

Hey @spookyvision

Why not use directly podman Podman Facts ?

[spookyvision]

Hey @spookyvision

Why not use directly podman Podman Facts ?

• because there are no podman operations - and neither should there be, IMHO. Podman is supposed to be a drop-in replacement, hence I would also tend to suggest that podman facts be deprecated in favor of more flexible docker facts (docker_executable setting)
• Docker facts support a lot more functionality than Podman facts. E.g. I need a list of containers, this is only supported by Docker facts.

[spookyvision]

after giving it some more thought, I think you raise an important point. If we went ahead with my original suggestion, then pyinfra would merely list "Docker facts/operations", and Podman users would likely come to the wrong conclusion that pyinfra does not support Podman. What I would really like to see is feature parity between Docker and Podman in pyinfra, and how that's achieved is an implementation detail. I'll rephrase my issue title + text accordingly.

[Kos]

Would it make sense to make the existing docker module configurable with a path to the docker binary? Then it would be very simple to do things like

docker.image(
    name="Pull nginx image",
    image="nginx:alpine",
    docker_binary="podman"
)

I can see 16 places where docker is hardcoded in operations/util/docker.py and looks like replacing that with podman is sufficient to make the basic use cases work (just tested), but I don't know for sure if it's really a 1-1 drop-in.

[Fizzadar]

Specifying binary is a good shout, might also be nice to provide podman.X operations that effectively just call docker.X with the flag set. This means they can be easily discovered in the docs/etc.

[spookyvision]

yeah, I initially suggested to make the docker binary configurable (and remove podman.*), but then podman support is pretty hard to discover. Explicit is better than implicit 🙃

[Kos]

Circling back after one week - I now realize that I probably don't want to manage podman containers with pyinfra anyway, instead I'd use pyinfra to create a systemd service for managing it. With that in mind, full feature parity is not as important for me as I had thought (which is good because trying to replicate the whole signature of podman run as python kwargs would be... a lot)

Edit: The recommended way seems to be podman-quadlet, leaving the link here for others.

[spookyvision]

(somewhat offtopic)
I can only semi-recommend quadlets. Socket activation performance is very nice and the main reason I'm running traefik as quadlet, however:

• with configuration-heavy services like traefik the unit files become brutally unreadable. Probably best to treat them as "compiled output format" - which ties into the other problem I keep running into:
• it's not fun to write them by hand, but they often can't be autogenerated from compose files either, because there's no 1:1 correspondence in features (unit files let you directly use secrets and percent-variables like %t (important for e.g. passing the podman socket on to containers that need it)

so ultimately I'd probably manage quadlets with some tool ... e.g. pyinfra.

[romain-dartigues]

By the way, there is a large overlaps between Docker / Podman and lxd / incus;
I think it can be possible to share some code (confer code from #1368).