feat: Ability to log WSGI environ in JSON logs, log_extra utility#55
feat: Ability to log WSGI environ in JSON logs, log_extra utility#55
environ in JSON logs, log_extra utility#55Conversation
src/common/gunicorn/logging.py
Outdated
| from common.gunicorn.utils import get_extra | ||
|
|
||
| _wsgi_extra_key_regex = re.compile( | ||
| r"^{(?P<key>[^}]+)}(?P<suffix>[%s])$" % "".join(WSGI_EXTRA_SUFFIX_TO_CATEGORY) |
There was a problem hiding this comment.
Do we need to perform this validation ourselves? If someone passes in a bad specification, it shouldn't be up to us to validate that IMO. This should blow up as early/violently as possible if something wrong was passed in.
There was a problem hiding this comment.
Do we need to perform this validation ourselves?
Unfortunately so as the atom templates are hard-coded in Gunicorn — the good news is they are part of user-facing API so very unlikely to change.
This should blow up as early/violently as possible if something wrong was passed in.
Currently, bad keys in the setting are just ignored by the logger. It's easy to make this pattern public and add validation in the main repo's common settings, though 👍
There was a problem hiding this comment.
What I meant to say is, if we can't make Gunicorn fail when using invalid templates, then we should respect its behaviour, even if that's silently doing nothing. In future Gunicorn versions this might change and it shouldn't be up to us to decide what the behaviour should be.
There was a problem hiding this comment.
Just to make sure we're on the same page, re.Pattern.match does not raise an exception when a match is not found.
There was a problem hiding this comment.
That's fine - my point is this code is not necessary, we don't need to validate things before they reach Gunicorn. It's up to users of this variable to pass the correct formats and ensure everything works as expected.
There was a problem hiding this comment.
Yeah, was just updating my comment to point out that the pattern is not used for validation, rather for convenient key/suffix extraction.
2 participants
This PR capitalises on Gunicorn's existing ability to log request/response headers and WSGI environ keys, enabling our JSON access log formatter to log these as well.
Flagsmith will now accept an
ACCESS_LOG_JSON_EXTRA_ITEMSsetting, consisting of Gunicorn atom keys, e.g.:{x-environment-key}i: theX-Environment-Keyrequest header...{content-type}o: theContent-Typeresponse header...{wsgi.version}e: thewsgi.versionWSGI environ value......in line with the
ACCESS_LOG_FORMATsetting we employ for rendering the non-JSON access logs.The PR also exposes the
common.gunicorn.utils.log_extrautility function which is a shortcut to logging the extra values as WSGI environ keys with theflagsmith.prefix. The custom key used for logging routes has been updated to match this convention.This is part 1 of an effort to enable logging environment keys/ids in access logs for a major customer.