`logging.shutdown()` should clear `logging._handlerList` if that's what it's called with

[cjw296]

Bug report

Bug description:

In the wild, I've encountered situations where logging.shutdown gets called more than once in a process.

The current implementation is called with logging._handlerList by default, meaning that handlers can have their flush() and close() methods called more than once, which can cause bad things to happen if those handlers are not amenable to being closed more than once.

If logging.shutdown() is processing logging._handlerList, it should clear that list once it finished to prevent handlers having their flush() and close() methods called more than once.

CPython versions tested on:

3.12, CPython main branch

Operating systems tested on:

Linux, macOS

Linked PRs

• gh-129268: Don't attempt to cleanup handlers again if logging.shutdown is called multiple times #129681

[ZeroIntensity]

Would you like to submit a PR?

[cjw296]

Would be good to get a second opinion as to why this is the way it is. @vsajip - you around on here?

[vsajip]

Well, the API documentation for logging.shutdown says:

This should be called at application exit and no further use of the logging system should be made after this call.

When the logging module is imported, it registers this function as an exit handler (see atexit), so normally there’s no need to do that manually.

So, calling shutdown() in user code is mildly questionable practice (as not needed, due to the atexit handler), and calling it more than once perhaps more strongly so (given the "no further use should be made ...").

logging.shutdown also catches and ignores OSError and ValueError exceptions raised when flushing/closing, so perhaps those bad things happening are due to handlers being badly written.

The logging.shutdown API is used internally in logging.config and in the logging test suite, so changing it could have ramifications there, too.

Since the case you mention involves calling shutdown more than once and perhaps involves handlers that don't just throw exceptions if called inappropriately, surely any changes to the stdlib in this area would be more of a band-aid, rather than addressing the root cause of the problems? I appreciate you may not have direct control of the code calling shutdown or of the handlers.

If the handlers throw some other stdlib exception than OSError and ValueError, I can look at broadening that catch clause.

[cjw296]

@vsajip - what reason could there be to not logging._handlerList.clear() once the processing is done during shutdown?

[picnixz]

Similar but not a duplicate: #129143

[vsajip]

what reason could there be to not

Don't know yet if there is one, but I'm generally not keen on making changes to cover for what might be bad practice in user code. I gave some specific reasons why I thought there might be bad practice, but those haven't been addressed.

As for #129143 - it just looks like a documentation inaccuracy, as the docstring matches the behaviour of the code.

[cjw296]

Okay, so I put up the PR, I guess where I'm at:

• I can see no downsides of this change.
• If .shutdown() is intended to only ever be called once, then it should not be part of the public API, but I think that ship has sailed.

So, hopefully a quick approval on the PR and we can all move on 😅

[vsajip]

• I'm not especially keen on making changes just because there're no immediately apparent downsides. If some emerge, I will have to deal with any fallout.
• Any function that is part of the API comes with conditions as to when to call it, how to call it etc. If those conditions aren't followed, that isn't the fault of the API.
• You still haven't addressed the points I made in my first response. As you can probably tell, I'm not especially keen on making this change, small though it is, as I consider you haven't given enough justification for it other than having to work around some code "out there" calling shutdown() more than once in a process, contravening the contract of the API as documented.