Skip to content

Commit 469f041

Browse files
chore(iast): fix circular imports with importlib_metadata [backport 2.21] (#12728)
Backport 3c9f4ed from #12711 to 2.21. If the IAST hook for importing modules is triggered before Telemetry (the other product that calls this function), a circular import can occur when calling `get_package_distributions` since it relies on `importlib_metadata`. To prevent this, since `get_package_distributions` caches the data, any subsequent calls won't invoke `importlib_metadata` again. For this reason, we execute this function before IAST runs. Error example https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-py/-/jobs/844601482 ``` Traceback (most recent call last): File "/app/benchmarks/.venv_baseline/lib/python3.9/site-packages/ddtrace/internal/module.py", line 257, in _find_first_hook callable(cond) and cond(module.__name__) File "/app/benchmarks/.venv_baseline/lib/python3.9/site-packages/ddtrace/appsec/_iast/_ast/ast_patching.py", line 484, in _should_iast_patch if _is_first_party(module_name): File "/app/benchmarks/.venv_baseline/lib/python3.9/site-packages/ddtrace/appsec/_iast/_ast/ast_patching.py", line 465, in _is_first_party _IMPORTLIB_PACKAGES = set(get_package_distributions()) File "/app/benchmarks/.venv_baseline/lib/python3.9/site-packages/ddtrace/internal/packages.py", line 63, in get_package_distributions _PACKAGE_DISTRIBUTIONS = _packages_distributions() File "/app/benchmarks/.venv_baseline/lib/python3.9/site-packages/ddtrace/internal/packages.py", line 325, in _packages_distributions pkg_to_dist[pkg].append(dist.metadata["Name"]) File "/app/benchmarks/.venv_baseline/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 499, in metadata return _adapters.Message(email.message_from_string(text)) AttributeError: partially initialized module 'importlib_metadata._adapters' has no attribute 'Message' (most likely due to a circular import) ``` This PR is a cherry-pick of one of the commits of this PR #12639 ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) Co-authored-by: Alberto Vara <[email protected]>
1 parent 502beb7 commit 469f041

File tree

1 file changed

+6
-0
lines changed

1 file changed

+6
-0
lines changed

ddtrace/appsec/_iast/__init__.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,12 @@ def enable_iast_propagation():
9292
global _iast_propagation_enabled
9393
if _iast_propagation_enabled:
9494
return
95+
# We need to preload the package_distributions because if we call importlib_metadata inside a module hook
96+
# it raises a circular-import
97+
from ddtrace.internal.packages import get_package_distributions
98+
99+
get_package_distributions()
100+
95101
log.debug("IAST enabled")
96102
ModuleWatchdog.register_pre_exec_module_hook(_should_iast_patch, _exec_iast_patched_module)
97103
_iast_propagation_enabled = True

0 commit comments

Comments
 (0)