plone-backend image is not responding correctly to SIGTERM

[fredvd]

While testing the cookiecutter-plone-starter docker compose setup today I noticed that when I stop the compose setup, the backend container takes 10-11 seconds to stop. This is the 10 second timeout when the application in side the container doesn't respond correclty to the 'nice' way of asking to stop (I think SIGTERM) and then a SIGKILL follows.

This might have to do something with our entrypoint script that starts the instancne and somewhere along the process chain the signals are not correctly passed along. Any takers? ;-)

[avoinea]

I'm not sure I can reproduce this one:

$ docker run --name plone -it --rm -p 8080:8080 -e SITE=Plone plone/plone-backend

docker top should give you the insights

 docker top plone 
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
500                 401333              401311              28                  14:29               pts/0               00:00:04            /app/bin/python /app/bin/runwsgi -v etc/zope.ini config_file=zope.conf

Installing htop in container also shows runwsgi as PID 1 👍

Doing exec CMD should usually be enough https://github.com/plone/plone-backend/blob/6.0.x/skeleton/docker-entrypoint.sh#L147

[fredvd]

@avoinea Thanks for checking. I was uncertain if I should post it here on plone-backend or if it maybe was a docker compose specific issue. But I can also reproduce it with the 'pure' plone-backend image .

If you run the container with:

$ docker run -d --name plone -it --rm -p 8080:8080 -e SITE=Plone plone/plone-backend

so it is started in the background. And when you then stop the container with

$ docker stop plone

[ 10 - 11 seconds]

$

You will see it takes 10-11 seconds before the container is really stopped.

Maybe when running the docker container in the foreground with stdout connected/redirected to the terminal session, the ctrl-c is passed on different and the container stops immediately. But running it detached, in compose or probably in a stack the signal is not coming through correctly.

[davisagli]

I think the signal is getting passed to the application fine but waitress doesn't have any support for a graceful shutdown using SIGTERM. See Pylons/waitress#198

Ctrl+C works because it sends SIGINT, not SIGTERM.

[davisagli]

As a workaround we could probably put STOPSIGNAL SIGINT in the Dockerfile.

But both that solution and the status quo have the downside of interrupting in-flight requests. It would be nice to use a WSGI server that supports graceful shutdown.

[Rudd-O]

Graceful shutdown is a must, seconding that request.

[davisagli]

This was enough to make it work in my project:

    import signal
    import sys

    signal.signal(signal.SIGTERM, lambda *a: sys.exit(0))

It is not a fully graceful shutdown because it can interrupt requests that are in progress. But it is an improvement: it shuts down immediately rather than delaying, and it gives a chance for resources to be closed and cleaned up properly, such as writing the relstorage persistent cache.