Skip to content

Commit d4e0ec0

Browse files
authored
fix(profiler): do not unload google.protobuf in module clean-up [backport #5747 to 1.13] (#5748)
Backport of #5747 to 1.13 When cleaning up the modules as part of the support for gevent, we also unload google.protobuf. However, starting with version 4.21, the new default upb backend does not tolerate being unloaded, which resulted in segfaults in applications with a runtime dependency on protobuf. ## Checklist - [x] Change(s) are motivated and described in the PR description. - [x] Testing strategy is described if automated tests are not included in the PR. - [x] Risk is outlined (performance impact, potential for breakage, maintainability, etc). - [x] Change is maintainable (easy to change, telemetry, documentation). - [x] [Library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines) are followed. - [x] Documentation is included (in-code, generated user docs, [public corp docs](https://github.com/DataDog/documentation/)). - [x] PR description includes explicit acknowledgement/acceptance of the performance implications of this PR as reported in the benchmarks PR comment. ## Reviewer Checklist - [x] Title is accurate. - [x] No unnecessary changes are introduced. - [x] Description motivates each change. - [x] Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [x] Testing strategy adequately addresses listed risk(s). - [x] Change is maintainable (easy to change, telemetry, documentation). - [x] Release note makes sense to a user of the library. - [x] Reviewer has explicitly acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment.
1 parent 5f327fa commit d4e0ec0

File tree

3 files changed

+34
-1
lines changed

3 files changed

+34
-1
lines changed

ddtrace/bootstrap/sitecustomize.py

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,18 @@ def drop(module_name):
121121
# uses a copy of that module that is distinct from the copy that user code
122122
# gets when it does `import threading`. The same applies to every module
123123
# not in `KEEP_MODULES`.
124-
KEEP_MODULES = frozenset(["atexit", "ddtrace", "asyncio", "concurrent", "typing", "logging", "attr"])
124+
KEEP_MODULES = frozenset(
125+
[
126+
"atexit",
127+
"ddtrace",
128+
"asyncio",
129+
"concurrent",
130+
"typing",
131+
"logging",
132+
"attr",
133+
"google.protobuf", # the upb backend in >= 4.21 does not like being unloaded
134+
]
135+
)
125136
if PY2:
126137
KEEP_MODULES_PY2 = frozenset(["encodings", "codecs"])
127138
for m in list(_ for _ in sys.modules if _ not in LOADED_MODULES):
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
fixes:
3+
- |
4+
profiler: Fixed a bug that caused segmentation faults in applications that
5+
use protobuf as a runtime dependency.

tests/profiling/test_main.py

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,3 +124,20 @@ def test_memalloc_no_init_error_on_fork():
124124
if not pid:
125125
exit(0)
126126
os.waitpid(pid, 0)
127+
128+
129+
@pytest.mark.subprocess(
130+
ddtrace_run=True,
131+
env=dict(
132+
DD_PROFILING_ENABLED="1",
133+
DD_UNLOAD_MODULES_FROM_SITECUSTOMIZE="1",
134+
),
135+
out="OK\n",
136+
err=None,
137+
)
138+
def test_profiler_start_up_with_module_clean_up_in_protobuf_app():
139+
# This can cause segfaults if we do module clean up with later versions of
140+
# protobuf. This is a regression test.
141+
from google.protobuf import empty_pb2 # noqa
142+
143+
print("OK")

0 commit comments

Comments
 (0)