Retain argv + fds on re-exec, fix USR2 under systemd by notifying new PID

[pajod]

This PR stops mangling sys.argv and socket fd numbers on re-exec, thus permitting much improved integration with systemd.

• While not meant to break anything before that, this patch is only effective in cPython >= 3.10.
  • Python docs explicitly say that relaunching with -m switch is the recommended procedure, so that should work

diff --git a/gunicorn/arbiter.py b/gunicorn/arbiter.py
-        args = sys.argv[:]
-        args.insert(0, sys.executable)
+        if sys.version_info < (3, 10):
+            args = sys.argv[:]
+            args.insert(0, sys.executable)
+        else:
+            args = sys.orig_argv[:]

• New feature: send timestamp to properly synchronize reload cycles using Type=notify-reload (systemd >= v253)

• Fixes: module invocation breaks USR2 upgrading, as modified argv results in modified path #3281

• Fixes: more status to systemd #1948

• Fixes: Bind to unix socket *and* IP: gunicorn only binds to socket. #2104

  • insofar as the surprising behavior is documented at least

• Fixes: Compatibility of systemd with zero downtime reloads with SIGUSR2 #2856

  • insofar as systemctl -s USR2 gunicorn.service now automatically issues graceful quit
  • systemctl reload gunicorn.service and systemctl restart gunicorn.service unchanged in .service example

• Improves systemd-specific case of: Error instead of warning in logs when sending SIGTERM since v21 #3050

• Improves sock fd leak pointed out in: fd leak in thread worker causing periodic outages #3339

• Follow-Up plans not included here:

  • the choice between announcing new MAINPID as early or as late as possible
    • ideally, we would not trigger the systemd[1]: gunicorn.service: Supervising process 1337 which is not our child. We'll most likely not notice when it exits. warning, at all
  • document difference between behavior of < 3.10 and Python >= 3.10
  • could be improved with BARRIER=1 so reloading/re-execing in quick success is reliable as well
  • could be improved with EXTEND_TIMEOUT_USEC= so automatic timeouts are reliable as well

[benoitc]

what , does thsi patch exactly in simple word and how do you guarantee the compatibility with current systems not running systemd?

[pajod]

This PR is a superset of #3419 and I will rebase and fix it once that one is merged.

what , does thsi patch exactly in simple word

Use fd numbers and argv as-is, and keep systemd informed about status changes.

how do you guarantee the compatibility with current systems not running systemd

The status changes are effectively no-op outside of systemd, so only 3 changes remain to be judged in that regard:

1. using original argv is the same except for one special case -- one probably nobody used, as suggested by me being 1st to complain about the crash
2. not duplicating file descriptors on re-exec and the fd:// setting -- getting exactly the same thing is the point of duplication. If that operation changed things, it would not really be a "duplicate"
3. extra warning -- only emitted on very clearly unintended configuration. also, its just one extra line logged per master startup, should not cause compat issues.

I deliberately did not guard these 3 changes behind a if systemd: branch, because they resolve issues that are merely unlikely to be encountered outside of systemd, not specific to systemd.