ambiguous relationship between "exec" command and "process" section

[ianpark]

"args" is a mandatory parameter of "process" section. This is valid if you use "run" or "create"+"start" commands.

But also, you can run "exec" command before starting a created container. What ever command you put next to 'exec', it's successfully executed with the all the parameters in "process" configuration. This is really useful to run multiple commands in a better control, and I don't HAVE TO use "start" to execute a command.

However I still need to fill "args" even though I will never call "start" command because it is a mandatory parameter. So this feature may mislead users to misuse "start" command for an alternative of "terminate". For instance, if you put a clean up script in "args" and call "start" when you finished your work (using "exec"), the container will be destroyed after cleaning up, which sounds like a nice graceful termination method.

What about making "args" optional, or separating out from "process" as "startingCommand" or something if possible?

[cyphar]

exec before start is meant to be used to tweak the setup of a container (for example, to create some binaries or hardlink some files). The reason why args is required is because a container has to have an init process that runs inside it. Now, when you're using exec as a way of running other processes, and you have no intention of running an init process inside the container, you're essentially misusing the interface. Sure, it'll work (kinda) but the fact that your init process is undefined by the specification (in runc's case, it's the runC binary itself while waiting for a message to come through a FIFO) will cause you issues.

Overall I personally don't think this is a use-case that makes sense in the context of containers. What does it mean to have a container with no processes inside it?

[ianpark]

I ended up with using exec, because there's no other way to run a dynamically populated command. config.jsonspecifies one command with all the fixtures, and no option to change the json file name. Of course it seems to work with service-like applications. But for some other purpose, "start" can be very ambiguous. Container is "stopped" as soon as the start command is finished. And then you will not be able to "exec" any further command. This interface behaviour largely narrows down the use cases.

Interestingly "start" command is not for starting the container itself, but for "pre-defined process". The life-cycle of a container has been already started as soon as "create" is called, and the parameters of "process" section are already applied. So calling "exec" on the created container should be valid action, and not calling "start" should be regarded as a valid use case IMHO.

[cyphar]

The process you specify in the config.json doesn't have to exist until start time. It's an explicit design decision that allows for dynamically changing what the actual container will run (though the actual argument format is unchanged).

Interestingly "start" command is not for starting the container itself, but for "pre-defined process". A container has been already started as soon as "create" is called, and the parameters of "process" section are already applied. So calling "exec" on the started container should be valid action, and not calling "start" should be regarded as a valid use case IMHO.

But the pre-defined process has not been started. The container has been created when you do create, but it hasn't been started until you run start. From an implementation standpoint, exec without start is not entirely safe. For example, PID namespaces don't make any sense without an init process (you literally cannot join a PID namespace if it has no PIDs inside it). So you have to have /some/ init process. But that init process can't be the process you specified (because you haven't told us to start it yet), so we have to run some code as the init process. And the only sane thing to do is to do nothing. But that now means that if you try to kill the container, you'll run into issues with signal propogation not working (just as though you'd used a shell as your init).

exec on a created container is a valid action, but using exec as an alternative to start (and then running an actual init process) is not something that I would recommend -- mainly because it means you won't know what is happening in the init process and confusion will occur. There's also the fact that parts of the spec don't make sense in such a setup (as you mention, process.args doesn't make much sense in such a setup) -- but the issue is not the spec but the setup IMO.

[wking]

On Mon, Sep 12, 2016 at 09:58:54AM -0700, Aleksa Sarai wrote:

But the pre-defined process has not been started. The container has
been created when you do create, but it hasn't been started
until you run start. From an implementation standpoint, exec
without start is not entirely safe. For example, PID namespaces
don't make any sense without an init process (you literally cannot
join a PID namespace if it has no PIDs inside it). So you have to
have /some/ init process. But that init process can't be the process
you specified (because you haven't told us to start it yet), so we
have to run some code as the init process.

I agree with all of this except “not entirely safe” (anything you
execute will be contained by the completed container). You just might
be surprised if you expect the (unspecified) init process to act a
certain way.

And the only sane thing to do is to do nothing.

This seems too strong for me. One reason for containers that you
create but never start is to manage shell-containers for nested
subcontainers 1.

exec on a created container is a valid action, but using
exec as an alternative to start (and then running an actual init
process) is not something that I would recommend…

The ‘create’ command will create the init process in the container's
PID namespace (if the container creates a PID namespace). I see no
way to ‘exec’ a new init process in that case. The closest you could
come would be to launch the systemd executable (or whatever) as PID 2
within the container's PID namespace. And I agree that is probably
not a good idea ;).

There's also the fact that parts of the spec don't make sense in
such a setup (as you mention, process.args doesn't make much sense
in such a setup) -- but the issue is not the spec but the setup IMO.

Fixed by the in-flight #489 ;).

[ianpark]

@cyphar @wking thanks for the comments, you really helped me out :-)
'create' creates the init process so I thought using 'exec' after it would be as safe as running the same command line by calling 'start'. I will start from this approach. thanks again guys!

[wking]

On Tue, Sep 13, 2016 at 01:11:19AM -0700, Ian Park wrote:

thanks again guys!

This sounds like the issue is closeable. If you're satisfied with the
spec as it stands or other open issues for covering your initial
confusion, then you shuold probably close this issue. If you think
there is other work (not yet in progress) which could help others
avoid the same confusion you had initially, we should keep this issue
open. But let us know what sort of changes you'd like to see ;).